-- Logs begin at Wed 2022-08-10 12:55:44 UTC, end at Wed 2022-08-10 13:57:36 UTC. -- Aug 10 13:23:47.238143 dv-cinder-852009-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 13:23:47.248612 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 13:23:47 2022] *** Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: nodename: dv-cinder-852009-3-iscsi Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: machine: x86_64 Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: clock source: unix Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: pcre jit disabled Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: detected number of CPU cores: 4 Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: current working directory: / Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: detected binary path: /usr/bin/uwsgi-core Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: your processes number limit is 47680 Aug 10 13:23:47.249187 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: your memory page size is 4096 bytes Aug 10 13:23:47.249928 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: detected max file descriptor number: 2048 Aug 10 13:23:47.249928 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: lock engine: pthread robust mutexes Aug 10 13:23:47.249928 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: thunder lock: enabled Aug 10 13:23:47.249928 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 13:23:47.249928 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 13:23:47.289633 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: Python main interpreter initialized at 0x555f5cb0c9c0 Aug 10 13:23:47.289633 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: python threads support enabled Aug 10 13:23:47.289633 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: your server socket listen backlog is limited to 100 connections Aug 10 13:23:47.289633 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: your mercy for graceful operations on workers is 90 seconds Aug 10 13:23:47.289973 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 13:23:47.290072 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: *** Operational MODE: preforking *** Aug 10 13:23:47.290360 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: *** uWSGI is running in multiple interpreter mode *** Aug 10 13:23:47.290360 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: spawned uWSGI master process (pid: 80113) Aug 10 13:23:47.290569 dv-cinder-852009-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 13:23:47.296375 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: spawned uWSGI worker 1 (pid: 80124, cores: 1) Aug 10 13:23:47.296375 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: spawned uWSGI worker 2 (pid: 80125, cores: 1) Aug 10 13:23:47.296375 dv-cinder-852009-3-iscsi devstack@keystone.service[80113]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 13:23:52.943308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 13:23:52.943308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: from cryptography.utils import int_from_bytes Aug 10 13:23:52.944121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 13:23:52.944121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: from cryptography.utils import int_from_bytes Aug 10 13:23:52.944543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 13:23:52.944543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: from cryptography.utils import int_from_bytes Aug 10 13:23:52.944658 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 13:23:52.944658 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: from cryptography.utils import int_from_bytes Aug 10 13:23:52.995424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 13:23:52.995654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 13:23:52.995799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] command line args: [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 13:23:52.996067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 13:23:52.996245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ================================================================================ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 13:23:52.996506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] admin_token = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.996754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] config_dir = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.996943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.997128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] config_source = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.997338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] debug = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.997543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.997942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.998119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.998354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.998472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 13:23:52.998599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.998734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 13:23:52.998836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.999057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_config_append = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.999245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.999398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] command line args: [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 13:23:52.999488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_dir = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.999607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_file = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:52.999820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 13:23:52.999943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.000036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ================================================================================ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 13:23:53.000208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.000606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] admin_token = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.000741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.000834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] config_dir = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.000927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] config_source = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] debug = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.001951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.002036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.002121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.002442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.002618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.002618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.002917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.003057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.003215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.003366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.003532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_config_append = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.003532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.003836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.004003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_dir = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.004003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_file = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.004296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.004467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.004630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] publish_errors = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.004801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.004963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.004963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.005293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.005442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.005604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.005764 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.005890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] standard_threads = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_journal = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.006909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.007026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_json = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.007026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_stderr = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.007233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.007354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_syslog = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.007465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.007581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.007581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.008321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.008444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.008556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.008686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.008799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.008910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.009072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.009187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.009187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.009425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.009542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.009542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.009542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.009865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.009865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] publish_errors = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.010075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.010194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.010306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.010417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.010528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.010637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.010756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.010879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.011033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.011150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.011321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] standard_threads = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.011443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.011604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.011727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.011922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.012043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.012162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.012274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_journal = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.012391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.012514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_json = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.012633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_stderr = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.012744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.012857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_syslog = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.012969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.013080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 13:23:53.013199 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.013347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.013584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.013700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.013971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.014097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.014328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.014459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.014655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.014782 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.014960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.015076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.015267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.015382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.015578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.015693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.015894 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.016016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.016215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.016332 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.016543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.016659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.016873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.016988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.017171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.017366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.017543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.017714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.017864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.018063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.018204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.018340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.018476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.018612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.018762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.018914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.019061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.019178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.019329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.019494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.019657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.019837 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.020018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.020200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.020367 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.020530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.020697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.020879 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.021059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.021271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.021436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.021600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.021766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.021956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.022135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.022313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.022480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.022647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.022820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.023100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.023270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.023452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.024281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.024446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.024621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.024805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.024973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.025147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.025392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.025566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.025730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.025910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.026080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.026237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.026398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.026568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.026734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.026906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.027065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.027221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.027370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.027526 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.027674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.027850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.028003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.028161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.028325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.028478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.028683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.028821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.028939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.029057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.029274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.029457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.029642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.029843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.030026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.030205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.030388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.030569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.030757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.030956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.031161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.031342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.031524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.031691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.031863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.032024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.032209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.032392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.032569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.032753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.032925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.033088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.033317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.033486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.033654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.033821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.033994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.034168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.034360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.034524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.034695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.034866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.035864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.036130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.036244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.036356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.036356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.036566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.036687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.036800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.036912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.037060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.037175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.037310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.037437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.037552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.037665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.037782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.037901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.038016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.038136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.038250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.038362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.038474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.038593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.038706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.038896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.039958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.external = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.040997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.041121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.041250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.041366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.password = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.041508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.041622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.token = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.041734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.041857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.041978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.042091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.042204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.042316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.042428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.042548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.042663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.042776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.042889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.043861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.044082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.044082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.044387 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.044548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.044662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.044775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.044888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.external = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.045855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.password = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.token = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.046988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.047109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.047233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.047357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.047474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.047588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.047708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.047831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.047946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.048974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.049932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.050028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.050124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.050226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.050324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.050437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.050536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.051646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.051762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.051878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.051993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.052121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.052239 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.052354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.052469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.052584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.052699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.052826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.052942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.053055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.053171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.053320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.053636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.053752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.053867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.053982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.054097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.054219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.054335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.054456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.054572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.054698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.054818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.054941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.055059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.055168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.055335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.055451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.055565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.055692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.055808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.055923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.056967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.057190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.057319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.057319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.057319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.057319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.057319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.057869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.057985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.058099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.058212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.058326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.058447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.058562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.058675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.058790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.058903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.059933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.060049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.060163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.060276 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.060389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.060511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.060697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.060821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.060936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.061056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.061173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.061173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.061491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.061491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.061712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.061826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.061939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.062984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.063903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.064933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.065146 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.065278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.065410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.065511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.065631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.065746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.065860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.065975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.066976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067763 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.067876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068310 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.068973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.069053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.069246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.069374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.069492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.069607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.069720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.069720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.069917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.070022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.070121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.070217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.070806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.070911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.071897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.072899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.073860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.074945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] role.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075239 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] role.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.075908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] role.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] role.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.076997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.077131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.077208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.077339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.077454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.077569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.077683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.077812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.077931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.078956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.079069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.079191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.079314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.079429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.079543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.079657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.079771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.079893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.080929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.081999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.082115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.082229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.082343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.082456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.082578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.082693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.082807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.082921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083387 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.083966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.084994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.085108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.085223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.085365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.085482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.085596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.085710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.085824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.085949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.086108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.086735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.086850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.086964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.087077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.087191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.087315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.087431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.087544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.087657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.087771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.087884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088489 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.088952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.089075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.089199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.089333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.089450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.089563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.089676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.089801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.089916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.090946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091295 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.091988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.092109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.092223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 13:23:53.092336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.092336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.092336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.092336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.092336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 13:23:53.092336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 13:23:53.118477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.118687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.118858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.118949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.119028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.119196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.129148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.129414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.159789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.159985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.160130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.160213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.215691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.216283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.269010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.269266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.269450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.269676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.339391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.339587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.339675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.339780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.339860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.340000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.340130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.340215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.340293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.340425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.340510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.340639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.365073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.365324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.365549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.365635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.365766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.365847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.365978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.366069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.366186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.366269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.366393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.366476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.366664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.366883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.378415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.379084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.392394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.392613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.392795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.392881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.392951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.393919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 13:23:53.410998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.411202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 13:23:53.411594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.411680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.411850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 13:23:53.411983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 13:23:53.412148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.412291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.412430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 13:23:53.412580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.412716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.412854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.412993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.413129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.413304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.413464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.424667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.424953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.425039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.425218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.425357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.425518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.425632 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.425919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.433115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.433477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.436213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.436494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.436607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.436792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.436895 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.437034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.437148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.437337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.437463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.437580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.437748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.437832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.438010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.438090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.438281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.438366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.438557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.438642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.438828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.438914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.439104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.439371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.462742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.462962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.464240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.464495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.464611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.464790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.469187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.469439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.469523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.469737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.469858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.470013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.476463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.476729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.476963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.477255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.480822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.481110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.481382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.481475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.481684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.481781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.482198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.482295 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.482479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.482565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.482754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.482849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.483015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.483111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.483316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.483596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.499717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.500001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.500155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.500452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.504020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.504471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.507193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.507488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.507666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.508513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.512553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.512982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.515735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.516207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.518895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.519192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.519414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.519497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.519718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.519799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.519997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.520078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.520271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.520354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.520552 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.520843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.536240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.536574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.536920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.537000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.537220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.537372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.537576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.537847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.544320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.544945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.545918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.546195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.546547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.546832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.549361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.549649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.549732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.549939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.550021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.550224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.550306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.550494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.550578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.550813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.563011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.563304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.563587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.563769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.563901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.564103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.564235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.564404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.564523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.564685 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.564814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.565076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.565305 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.565736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.582166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.585211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.588258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.589883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.591396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.591688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.591958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.592047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.597151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.598481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.600120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.602664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.603136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.603428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.603714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.603994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.604311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.605872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.606171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.606468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.612708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.613010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.613221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.613451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.614885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.618395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.618690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.618984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.619260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.620846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.621142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.621461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.621743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.622018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.622294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.622570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.622850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.623245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.623528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.623803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.624068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.624345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.624618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.627565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.627862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.628144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.635820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 13:23:53.636109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.636392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.636732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.637015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.637333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.637613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.637887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.638272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.638556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.638831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.639097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.639383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.639656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 13:23:53.658859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555f5cb0c9c0 pid: 80125 (default app) Aug 10 13:23:53.662343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4caf78-710b-4687-b289-f278955c014c None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:53.662627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4caf78-710b-4687-b289-f278955c014c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:53.662899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4caf78-710b-4687-b289-f278955c014c None None] PATH_INFO: `/v3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:53.667788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Aug 10 13:23:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:23:53.673653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555f5cb0c9c0 pid: 80124 (default app) Aug 10 13:24:01.426482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129d61a-4d84-4f55-8395-ab2f0c1a7b6c None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:01.427062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129d61a-4d84-4f55-8395-ab2f0c1a7b6c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:01.427062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129d61a-4d84-4f55-8395-ab2f0c1a7b6c None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:01.428323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 2/2] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:01.436071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:01.436544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:01.436822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:01.508228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: INFO dbcounter [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] Registered counter for database keystone Aug 10 13:24:01.528365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG oslo_db.sqlalchemy.engines [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80124) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 13:24:01.534790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writer thread running {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 13:24:01.721403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG passlib.handlers.bcrypt [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80124) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 13:24:01.721812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG passlib.handlers.bcrypt [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80124) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 13:24:01.779703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:01.781220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:01.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1/3] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 13:24:02.069220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129d61a-4d84-4f55-8395-ab2f0c1a7b6c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:02.141077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: INFO dbcounter [None req-e129d61a-4d84-4f55-8395-ab2f0c1a7b6c None None] Registered counter for database keystone Aug 10 13:24:02.162570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG oslo_db.sqlalchemy.engines [None req-e129d61a-4d84-4f55-8395-ab2f0c1a7b6c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80125) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 13:24:02.169643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writer thread running {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 13:24:02.289694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129d61a-4d84-4f55-8395-ab2f0c1a7b6c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:02.294014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85f071-3f17-46d1-87ea-0634ee0ae164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:02.294640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85f071-3f17-46d1-87ea-0634ee0ae164 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:02.294918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85f071-3f17-46d1-87ea-0634ee0ae164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:02.295185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85f071-3f17-46d1-87ea-0634ee0ae164 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:02.303928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85f071-3f17-46d1-87ea-0634ee0ae164 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:02.356065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9f85f071-3f17-46d1-87ea-0634ee0ae164 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 13:24:02.356323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85f071-3f17-46d1-87ea-0634ee0ae164 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:02.367008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 3/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Aug 10 13:24:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:24:02.373102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:02.387716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751397c-f8ed-4e44-afe9-4d1b1003e0e7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:02.392133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fe99e-ccb0-4af2-9a6b-37387de67703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:02.393341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe99e-ccb0-4af2-9a6b-37387de67703 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:02.393627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe99e-ccb0-4af2-9a6b-37387de67703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:02.393895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe99e-ccb0-4af2-9a6b-37387de67703 admin admin] PATH_INFO: `/v3/projects/e7cfe98f92774b529ebade94e29001a3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:02.395366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fe99e-ccb0-4af2-9a6b-37387de67703 admin admin] RBAC: Authorizing `identity:get_project(project_id=e7cfe98f92774b529ebade94e29001a3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:02.445774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cd4fe99e-ccb0-4af2-9a6b-37387de67703 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 13:24:02.446728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fe99e-ccb0-4af2-9a6b-37387de67703 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:02.450810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 2/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Aug 10 13:24:02 2022] GET /identity/v3/projects/e7cfe98f92774b529ebade94e29001a3 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:04.460529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1993d-3e69-4cb1-b41d-9f0b2eec03fb None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:04.461042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1993d-3e69-4cb1-b41d-9f0b2eec03fb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:04.461126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1993d-3e69-4cb1-b41d-9f0b2eec03fb None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:04.465172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 4/6] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:04.470887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a566-b72f-4283-8b0c-caafdb6bd81d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:04.471356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a566-b72f-4283-8b0c-caafdb6bd81d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:04.471449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a566-b72f-4283-8b0c-caafdb6bd81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:04.494271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1d36a566-b72f-4283-8b0c-caafdb6bd81d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:04.495815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1d36a566-b72f-4283-8b0c-caafdb6bd81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:04.503898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 3/7] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 13:24:04.525933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1993d-3e69-4cb1-b41d-9f0b2eec03fb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:04.537950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1993d-3e69-4cb1-b41d-9f0b2eec03fb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:04.542666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a43b0-8b5a-4a81-8f69-ef9f0c7d0241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:04.543278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a43b0-8b5a-4a81-8f69-ef9f0c7d0241 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:04.543555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a43b0-8b5a-4a81-8f69-ef9f0c7d0241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:04.543822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a43b0-8b5a-4a81-8f69-ef9f0c7d0241 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:04.552401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a43b0-8b5a-4a81-8f69-ef9f0c7d0241 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:04.554241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a43b0-8b5a-4a81-8f69-ef9f0c7d0241 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:04.564200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 5/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Aug 10 13:24:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:24:04.570091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36a566-b72f-4283-8b0c-caafdb6bd81d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:04.579368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36a566-b72f-4283-8b0c-caafdb6bd81d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:04.583731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685fc68-b3b5-452d-ad21-2f1c3de3223b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:04.584351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685fc68-b3b5-452d-ad21-2f1c3de3223b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:04.584626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685fc68-b3b5-452d-ad21-2f1c3de3223b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:04.584894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685fc68-b3b5-452d-ad21-2f1c3de3223b admin admin] PATH_INFO: `/v3/users/1d95623897cc4c628d91416fe290bfa2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:04.586750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685fc68-b3b5-452d-ad21-2f1c3de3223b admin admin] RBAC: Authorizing `identity:get_user(user_id=1d95623897cc4c628d91416fe290bfa2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:04.588582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685fc68-b3b5-452d-ad21-2f1c3de3223b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:04.590223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 4/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Aug 10 13:24:04 2022] GET /identity/v3/users/1d95623897cc4c628d91416fe290bfa2 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:06.678175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce3069-6433-42f1-9fdb-fabc19233473 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:06.678770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce3069-6433-42f1-9fdb-fabc19233473 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:06.678770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce3069-6433-42f1-9fdb-fabc19233473 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:06.680303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 6/10] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:06.687814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d7d0-0fee-4a46-b2ff-12c50dd29cbc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:06.688333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d7d0-0fee-4a46-b2ff-12c50dd29cbc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:06.688414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d7d0-0fee-4a46-b2ff-12c50dd29cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:06.707158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-bd42d7d0-0fee-4a46-b2ff-12c50dd29cbc None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:06.708355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bd42d7d0-0fee-4a46-b2ff-12c50dd29cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:06.753060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d908c-f2ae-4818-8ac3-fcf320468fad None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:06.753558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d908c-f2ae-4818-8ac3-fcf320468fad None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:06.754007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d908c-f2ae-4818-8ac3-fcf320468fad None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:06.756031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 7/11] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:06.762809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:06.763244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:06.763697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:06.794497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 5/12] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:06.813789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG passlib.handlers.bcrypt [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80125) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 13:24:06.814211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG passlib.handlers.bcrypt [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80125) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 13:24:06.818611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42d7d0-0fee-4a46-b2ff-12c50dd29cbc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:06.848112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:06.861877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:06.863514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:06.932251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42d7d0-0fee-4a46-b2ff-12c50dd29cbc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:06.967536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 8/13] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:06.967838 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f62793-440c-4fdc-b87f-c15df5ded861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:06.968482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62793-440c-4fdc-b87f-c15df5ded861 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:06.968763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62793-440c-4fdc-b87f-c15df5ded861 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:06.969037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62793-440c-4fdc-b87f-c15df5ded861 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:06.981313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f62793-440c-4fdc-b87f-c15df5ded861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:06.990144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-58f62793-440c-4fdc-b87f-c15df5ded861 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:06.990934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:06.992402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:06.993214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 6/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:06.999003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f62793-440c-4fdc-b87f-c15df5ded861 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:07.094920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:07.095656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574ad39-4762-4aa4-9960-e530c2e30f63 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:07.098534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:07.125412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f62793-440c-4fdc-b87f-c15df5ded861 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:07.138954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ab4d9-01e5-44a3-818a-89fa60ffa9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:07.139828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab4d9-01e5-44a3-818a-89fa60ffa9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:07.140114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab4d9-01e5-44a3-818a-89fa60ffa9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:07.140385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab4d9-01e5-44a3-818a-89fa60ffa9a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:07.166283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:07.167618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:07.167905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:07.168177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:07.168854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:07.169152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:07.170585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:07.191618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 7/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:07.197343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:07.214599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ab4d9-01e5-44a3-818a-89fa60ffa9a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:07.216150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ab4d9-01e5-44a3-818a-89fa60ffa9a6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:07.340500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 9/16] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:24:07.346545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ab4d9-01e5-44a3-818a-89fa60ffa9a6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:07.349828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:07.419555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c160b22-42d2-46f3-acd3-0b7a23f7012f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:07.454717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47afc-443b-4feb-b11c-4a3dbc8f4f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:07.455291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47afc-443b-4feb-b11c-4a3dbc8f4f39 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:07.455563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47afc-443b-4feb-b11c-4a3dbc8f4f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:07.455831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47afc-443b-4feb-b11c-4a3dbc8f4f39 None admin] PATH_INFO: `/v3/users/1d95623897cc4c628d91416fe290bfa2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:07.456951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc47afc-443b-4feb-b11c-4a3dbc8f4f39 None admin] RBAC: Authorizing `identity:get_user(user_id=1d95623897cc4c628d91416fe290bfa2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:07.458562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc47afc-443b-4feb-b11c-4a3dbc8f4f39 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:07.460171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 8/17] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:07 2022] GET /identity/v3/users/1d95623897cc4c628d91416fe290bfa2 => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:07.465752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47afc-443b-4feb-b11c-4a3dbc8f4f39 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:07.469765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ab4d9-01e5-44a3-818a-89fa60ffa9a6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:07.481195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:07.505044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:07.505957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:07.506236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:07.506507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:07.525216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:07.526740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:24:07.545692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:07.546651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:24:07.547448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:24:07.547448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:24:07.547448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:07.547448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:07.552723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:07.600532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47afc-443b-4feb-b11c-4a3dbc8f4f39 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:07.627148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:07.643805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb13b6-2a03-4fd3-99d3-904499ff4e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:07.644551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb13b6-2a03-4fd3-99d3-904499ff4e1c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:07.644859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb13b6-2a03-4fd3-99d3-904499ff4e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:07.645127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb13b6-2a03-4fd3-99d3-904499ff4e1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:07.647611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb13b6-2a03-4fd3-99d3-904499ff4e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:07.649082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb13b6-2a03-4fd3-99d3-904499ff4e1c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:07.650715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:07.657167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb13b6-2a03-4fd3-99d3-904499ff4e1c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:07.673584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414498a-8c10-4d55-ac18-f5b5d5898ac3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:07.715715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f793-5666-42d3-bfda-078c54948aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:07.716357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f793-5666-42d3-bfda-078c54948aea None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:07.716592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f793-5666-42d3-bfda-078c54948aea None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:07.716777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f793-5666-42d3-bfda-078c54948aea None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:07.717459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5f793-5666-42d3-bfda-078c54948aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:07.717760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5f793-5666-42d3-bfda-078c54948aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:07.719149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5f793-5666-42d3-bfda-078c54948aea None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:07.737445 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:07.747963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:07.771610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb13b6-2a03-4fd3-99d3-904499ff4e1c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:07.808811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83053eec-7110-46b1-8d8b-078bec5c9ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:07.809473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83053eec-7110-46b1-8d8b-078bec5c9ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:07.809752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83053eec-7110-46b1-8d8b-078bec5c9ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:07.810017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83053eec-7110-46b1-8d8b-078bec5c9ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:07.810751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83053eec-7110-46b1-8d8b-078bec5c9ee5 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.domain.id=default, user.id=1d95623897cc4c628d91416fe290bfa2) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:07.811051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83053eec-7110-46b1-8d8b-078bec5c9ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:07.812520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83053eec-7110-46b1-8d8b-078bec5c9ee5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:07.820004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Aug 10 13:24:07 2022] GET /identity/v3/role_assignments?user.id=1d95623897cc4c628d91416fe290bfa2&scope.domain.id=default&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 233 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:10.396350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09795-04f6-416d-88a3-6d0dcc6c0eb2 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:10.397105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09795-04f6-416d-88a3-6d0dcc6c0eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:10.397105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09795-04f6-416d-88a3-6d0dcc6c0eb2 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:10.399059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:10.416316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a5877-98ac-4e2a-b78e-0e8659e78e26 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:10.417797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a5877-98ac-4e2a-b78e-0e8659e78e26 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:10.418543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a5877-98ac-4e2a-b78e-0e8659e78e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:10.459069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2a0a5877-98ac-4e2a-b78e-0e8659e78e26 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:10.462171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2a0a5877-98ac-4e2a-b78e-0e8659e78e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:10.571667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:10.606252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b09795-04f6-416d-88a3-6d0dcc6c0eb2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:10.717932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a477ee-d3b2-4cde-b32c-4084bd669192 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:10.718341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a477ee-d3b2-4cde-b32c-4084bd669192 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:10.718516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a477ee-d3b2-4cde-b32c-4084bd669192 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:10.719809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:10.746791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873113a-881e-4f41-a9b1-ffb74a2dd0a0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:10.747209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873113a-881e-4f41-a9b1-ffb74a2dd0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:10.747472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873113a-881e-4f41-a9b1-ffb74a2dd0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:10.760444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b09795-04f6-416d-88a3-6d0dcc6c0eb2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:10.807033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b873113a-881e-4f41-a9b1-ffb74a2dd0a0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:10.809063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b873113a-881e-4f41-a9b1-ffb74a2dd0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:10.809847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:10.831145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694ee1e-6d7b-4890-bfe7-c641a4cc4316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:10.831672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694ee1e-6d7b-4890-bfe7-c641a4cc4316 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:10.831937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694ee1e-6d7b-4890-bfe7-c641a4cc4316 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:10.832204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694ee1e-6d7b-4890-bfe7-c641a4cc4316 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:10.832887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694ee1e-6d7b-4890-bfe7-c641a4cc4316 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:10.834305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694ee1e-6d7b-4890-bfe7-c641a4cc4316 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:11.045803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 13/25] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 13:24:11.097686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873113a-881e-4f41-a9b1-ffb74a2dd0a0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:11.117733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:11.234402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 13/26] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 10 13:24:10 2022] POST /identity/v3/roles => generated 219 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:11.340256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873113a-881e-4f41-a9b1-ffb74a2dd0a0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:11.412586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290936-11ea-41c8-bb1d-fca7b5a95a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:11.414182 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290936-11ea-41c8-bb1d-fca7b5a95a0b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:11.414933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290936-11ea-41c8-bb1d-fca7b5a95a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:11.415653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290936-11ea-41c8-bb1d-fca7b5a95a0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:11.444411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22290936-11ea-41c8-bb1d-fca7b5a95a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-22290936-11ea-41c8-bb1d-fca7b5a95a0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:11.455869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:11.457541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:11.459108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:11.463568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 14/27] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:11.468284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce67aa4-3bd7-4cd4-8223-93f77d0bd840 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:11.468284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce67aa4-3bd7-4cd4-8223-93f77d0bd840 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:11.468284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce67aa4-3bd7-4cd4-8223-93f77d0bd840 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:11.471159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290936-11ea-41c8-bb1d-fca7b5a95a0b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:11.473448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 14/28] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:11.483047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c9cc-76fe-4956-a88a-22699b12502e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:11.483331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c9cc-76fe-4956-a88a-22699b12502e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:11.483683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c9cc-76fe-4956-a88a-22699b12502e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:11.522566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bfe7c9cc-76fe-4956-a88a-22699b12502e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:11.524170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bfe7c9cc-76fe-4956-a88a-22699b12502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:11.603359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:11.643604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290936-11ea-41c8-bb1d-fca7b5a95a0b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:11.685189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:11.685900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:11.686197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:11.686450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:11.687118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:11.687413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:11.689012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:11.699166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 15/29] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:11.704709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:11.751448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 15/30] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:11.832385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b8ba0-59ca-46d7-86df-def1a5b960b2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:11.837605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7c9cc-76fe-4956-a88a-22699b12502e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:11.865197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:11.877258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f22b7-e772-4d3d-b23d-b7f53dc82d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:11.877978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f22b7-e772-4d3d-b23d-b7f53dc82d47 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:11.878321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f22b7-e772-4d3d-b23d-b7f53dc82d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:11.878655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f22b7-e772-4d3d-b23d-b7f53dc82d47 None admin] PATH_INFO: `/v3/users/1d95623897cc4c628d91416fe290bfa2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:11.879790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f22b7-e772-4d3d-b23d-b7f53dc82d47 None admin] RBAC: Authorizing `identity:get_user(user_id=1d95623897cc4c628d91416fe290bfa2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:11.881261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f22b7-e772-4d3d-b23d-b7f53dc82d47 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:11.883646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 16/31] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:11 2022] GET /identity/v3/users/1d95623897cc4c628d91416fe290bfa2 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:11.884542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f350eac-c88f-4ad3-a28b-aed2f41abbbb None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:11.884824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f350eac-c88f-4ad3-a28b-aed2f41abbbb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:11.885085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f350eac-c88f-4ad3-a28b-aed2f41abbbb None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:11.886658 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 17/32] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:11.893539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6becd74-e443-46de-983b-98f9dd88983e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:11.893858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6becd74-e443-46de-983b-98f9dd88983e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:11.894108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6becd74-e443-46de-983b-98f9dd88983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:11.917280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e6becd74-e443-46de-983b-98f9dd88983e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:11.918660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e6becd74-e443-46de-983b-98f9dd88983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:11.965884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7c9cc-76fe-4956-a88a-22699b12502e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:11.980196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:12.002783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 18/33] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:12.003637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6becd74-e443-46de-983b-98f9dd88983e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:12.037642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:12.038644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.039192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.039663 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.051447 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:12.053433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:24:12.071131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:12.072045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:24:12.072842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:24:12.072842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:24:12.072842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:12.074721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 16/34] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:12.075803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:12.153549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6becd74-e443-46de-983b-98f9dd88983e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:12.197018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:12.207745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52139633-b3b5-49f7-bf70-fa6d0629828a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:12.217030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292979b6-bb5c-4407-914e-93fad858939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:12.218517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292979b6-bb5c-4407-914e-93fad858939d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.219223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292979b6-bb5c-4407-914e-93fad858939d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.219906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292979b6-bb5c-4407-914e-93fad858939d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.222375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292979b6-bb5c-4407-914e-93fad858939d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:12.225304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292979b6-bb5c-4407-914e-93fad858939d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:12.229533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 19/35] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:12.231194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292979b6-bb5c-4407-914e-93fad858939d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:12.251478 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a87ca6-c16d-457d-a157-88b977562ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:12.252939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87ca6-c16d-457d-a157-88b977562ece None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.253512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87ca6-c16d-457d-a157-88b977562ece None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.254154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87ca6-c16d-457d-a157-88b977562ece None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.255395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a87ca6-c16d-457d-a157-88b977562ece None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:12.257680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a87ca6-c16d-457d-a157-88b977562ece None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:12.277164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 17/36] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 10 13:24:12 2022] POST /identity/v3/roles => generated 225 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:12.278686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a87ca6-c16d-457d-a157-88b977562ece None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:12.312179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:12.374257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292979b6-bb5c-4407-914e-93fad858939d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:12.429656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a87ca6-c16d-457d-a157-88b977562ece None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:12.433375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29ef9c-504e-48ce-8b59-3979e46298e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:12.434002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29ef9c-504e-48ce-8b59-3979e46298e7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.434278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29ef9c-504e-48ce-8b59-3979e46298e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.434548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29ef9c-504e-48ce-8b59-3979e46298e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.435211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29ef9c-504e-48ce-8b59-3979e46298e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:12.435507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29ef9c-504e-48ce-8b59-3979e46298e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:12.436930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29ef9c-504e-48ce-8b59-3979e46298e7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:12.463253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 20/37] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:12.464119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127634b8-9390-4e73-8368-ae46ce213503 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.464119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127634b8-9390-4e73-8368-ae46ce213503 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.464119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127634b8-9390-4e73-8368-ae46ce213503 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.467450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 21/38] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:12.470512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127634b8-9390-4e73-8368-ae46ce213503 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:12.488164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b9c25-6cac-42ba-9c90-6e7e451aa309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:12.488944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b9c25-6cac-42ba-9c90-6e7e451aa309 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.489319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b9c25-6cac-42ba-9c90-6e7e451aa309 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.489659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b9c25-6cac-42ba-9c90-6e7e451aa309 None admin] PATH_INFO: `/v3/domains/default/users/1d95623897cc4c628d91416fe290bfa2/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.492435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b9c25-6cac-42ba-9c90-6e7e451aa309 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1d95623897cc4c628d91416fe290bfa2, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:12.494651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b9c25-6cac-42ba-9c90-6e7e451aa309 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:12.513011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 18/39] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Aug 10 13:24:12 2022] PUT /identity/v3/domains/default/users/1d95623897cc4c628d91416fe290bfa2/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:12.514661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0f9ca-320d-401e-b7bf-8a3503e4a3eb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.515145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0f9ca-320d-401e-b7bf-8a3503e4a3eb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.515686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0f9ca-320d-401e-b7bf-8a3503e4a3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.537465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f3a0f9ca-320d-401e-b7bf-8a3503e4a3eb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:12.539675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f3a0f9ca-320d-401e-b7bf-8a3503e4a3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:12.555257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:12.628498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127634b8-9390-4e73-8368-ae46ce213503 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:12.645257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 19/40] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:12.645257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86699ff0-615e-4dad-ab6b-7203f981298e None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.645257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86699ff0-615e-4dad-ab6b-7203f981298e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.645257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86699ff0-615e-4dad-ab6b-7203f981298e None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.645643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 20/41] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:12.649059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c52b5-08a7-40de-9903-2578f789bf95 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.649372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c52b5-08a7-40de-9903-2578f789bf95 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.649645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c52b5-08a7-40de-9903-2578f789bf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.676507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-934c52b5-08a7-40de-9903-2578f789bf95 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:12.677749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-934c52b5-08a7-40de-9903-2578f789bf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:12.678148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5384-4fbd-4d43-947c-a38736322083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:12.678796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5384-4fbd-4d43-947c-a38736322083 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.679079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5384-4fbd-4d43-947c-a38736322083 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.679380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5384-4fbd-4d43-947c-a38736322083 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.680193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f5384-4fbd-4d43-947c-a38736322083 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:12.680633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:12.681741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f5384-4fbd-4d43-947c-a38736322083 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:12.767910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 21/42] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:12.769020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c52b5-08a7-40de-9903-2578f789bf95 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:12.822774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 22/43] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:24:12 2022] POST /identity/v3/projects => generated 301 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:12.823770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5384-4fbd-4d43-947c-a38736322083 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:12.882365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c52b5-08a7-40de-9903-2578f789bf95 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:12.927725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1aeb-58b0-48f5-b217-abdc925aaf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:12.928794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1aeb-58b0-48f5-b217-abdc925aaf09 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:12.929908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1aeb-58b0-48f5-b217-abdc925aaf09 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:12.930355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1aeb-58b0-48f5-b217-abdc925aaf09 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:12.931226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c1aeb-58b0-48f5-b217-abdc925aaf09 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:12.932935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c1aeb-58b0-48f5-b217-abdc925aaf09 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:12.945817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:12.953397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 22/44] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 10 13:24:12 2022] POST /identity/v3/roles => generated 223 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:12.958338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5384-4fbd-4d43-947c-a38736322083 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:12.962685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:13.014299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f064e-76b4-444c-a183-267daa6f2b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:13.015378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f064e-76b4-444c-a183-267daa6f2b50 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:13.015908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f064e-76b4-444c-a183-267daa6f2b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:13.016373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f064e-76b4-444c-a183-267daa6f2b50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:13.019059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f064e-76b4-444c-a183-267daa6f2b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:13.021116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f064e-76b4-444c-a183-267daa6f2b50 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:13.023870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 23/45] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:13.031070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1aeb-58b0-48f5-b217-abdc925aaf09 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:13.151231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1aeb-58b0-48f5-b217-abdc925aaf09 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:13.186804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b2897-0e8e-404e-ba66-3517d7850e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:13.187420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2897-0e8e-404e-ba66-3517d7850e39 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:13.187694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2897-0e8e-404e-ba66-3517d7850e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:13.187961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2897-0e8e-404e-ba66-3517d7850e39 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:13.188606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b2897-0e8e-404e-ba66-3517d7850e39 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:13.190083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b2897-0e8e-404e-ba66-3517d7850e39 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:13.219751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:24:13 2022] POST /identity/v3/projects => generated 312 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:14.614162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc979e27-9159-44ef-9ad3-45658220bd24 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:14.614724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc979e27-9159-44ef-9ad3-45658220bd24 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:14.614724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc979e27-9159-44ef-9ad3-45658220bd24 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:14.615960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 24/47] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:14.621080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f1ab1-1584-4684-8bad-20d4008402a2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:14.621522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f1ab1-1584-4684-8bad-20d4008402a2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:14.621657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f1ab1-1584-4684-8bad-20d4008402a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:14.639490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f30f1ab1-1584-4684-8bad-20d4008402a2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:14.640699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f30f1ab1-1584-4684-8bad-20d4008402a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:14.686053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:14.707527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 24/48] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:14.730538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc979e27-9159-44ef-9ad3-45658220bd24 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:14.836406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc979e27-9159-44ef-9ad3-45658220bd24 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:14.870327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f6ad6-b645-42dd-a811-651b365272c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:14.870876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f6ad6-b645-42dd-a811-651b365272c5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:14.871153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f6ad6-b645-42dd-a811-651b365272c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:14.871421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f6ad6-b645-42dd-a811-651b365272c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:14.881105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f6ad6-b645-42dd-a811-651b365272c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-be6f6ad6-b645-42dd-a811-651b365272c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:14.887032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:14.887996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:14.888803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 25/49] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:14.893955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f1ab1-1584-4684-8bad-20d4008402a2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:14.990118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f1ab1-1584-4684-8bad-20d4008402a2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:15.022893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:15.024038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:15.024315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:15.024581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:15.025261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:15.025557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:15.026939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:15.038425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 25/50] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:15.044047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f6ad6-b645-42dd-a811-651b365272c5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:15.077922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:15.150284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f6ad6-b645-42dd-a811-651b365272c5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:15.183273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1509c5-82a1-4af3-89df-2405a979d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:15.183896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1509c5-82a1-4af3-89df-2405a979d1dd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:15.184174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1509c5-82a1-4af3-89df-2405a979d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:15.184442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1509c5-82a1-4af3-89df-2405a979d1dd None admin] PATH_INFO: `/v3/users/1d95623897cc4c628d91416fe290bfa2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:15.185678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1509c5-82a1-4af3-89df-2405a979d1dd None admin] RBAC: Authorizing `identity:get_user(user_id=1d95623897cc4c628d91416fe290bfa2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:15.187126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1509c5-82a1-4af3-89df-2405a979d1dd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:15.188679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 26/51] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:15 2022] GET /identity/v3/users/1d95623897cc4c628d91416fe290bfa2 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:15.194113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:15.237016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:15.239755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bc34f-33c9-4a83-8275-864fa34504c5 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:15.240053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bc34f-33c9-4a83-8275-864fa34504c5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:15.240329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bc34f-33c9-4a83-8275-864fa34504c5 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:15.242251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 27/52] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:15.249694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b392b5-89c0-4647-a2dd-294fadbd8d22 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:15.249982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b392b5-89c0-4647-a2dd-294fadbd8d22 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:15.250424 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b392b5-89c0-4647-a2dd-294fadbd8d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:15.268880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-29b392b5-89c0-4647-a2dd-294fadbd8d22 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:15.270418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-29b392b5-89c0-4647-a2dd-294fadbd8d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:15.296492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daafc87-c7a8-4a4b-8f27-a4933b695b95 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:15.331805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85b48-aeb5-4b82-a7a8-1f62494e7987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:15.332456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85b48-aeb5-4b82-a7a8-1f62494e7987 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:15.332736 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85b48-aeb5-4b82-a7a8-1f62494e7987 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:15.333004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85b48-aeb5-4b82-a7a8-1f62494e7987 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:15.334132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca85b48-aeb5-4b82-a7a8-1f62494e7987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:15.335697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca85b48-aeb5-4b82-a7a8-1f62494e7987 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:15.337359 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 26/53] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:15.338002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 28/54] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:15.342830 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85b48-aeb5-4b82-a7a8-1f62494e7987 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:15.360641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b392b5-89c0-4647-a2dd-294fadbd8d22 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:15.390312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:15.443031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85b48-aeb5-4b82-a7a8-1f62494e7987 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:15.454695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:15.464915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b392b5-89c0-4647-a2dd-294fadbd8d22 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:15.478779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:15.479397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:15.479671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:15.479940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:15.480660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.domain.id=default, user.id=1d95623897cc4c628d91416fe290bfa2) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:15.480964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:15.482702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:15.490464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 27/55] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Aug 10 13:24:15 2022] GET /identity/v3/role_assignments?user.id=1d95623897cc4c628d91416fe290bfa2&scope.domain.id=default&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:15.499143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b3b58-bb40-495c-ae94-094956a6870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:15.499798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b3b58-bb40-495c-ae94-094956a6870c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:15.500076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b3b58-bb40-495c-ae94-094956a6870c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:15.500344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b3b58-bb40-495c-ae94-094956a6870c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:15.501583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b3b58-bb40-495c-ae94-094956a6870c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:15.503088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b3b58-bb40-495c-ae94-094956a6870c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:15.504745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 29/56] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:15.510914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:15.616297 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd7066-6414-4d96-869e-80d149b52af0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:15.651746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67582e1e-cff6-4f44-8e79-71044fe43499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:15.652383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67582e1e-cff6-4f44-8e79-71044fe43499 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:15.652639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67582e1e-cff6-4f44-8e79-71044fe43499 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:15.652909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67582e1e-cff6-4f44-8e79-71044fe43499 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:15.653585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67582e1e-cff6-4f44-8e79-71044fe43499 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:15.655058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67582e1e-cff6-4f44-8e79-71044fe43499 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:15.682921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 28/57] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Aug 10 13:24:15 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:17.670855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9b6d-495e-4dea-ae43-0a59d6522c9f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:17.671388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9b6d-495e-4dea-ae43-0a59d6522c9f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:17.671388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9b6d-495e-4dea-ae43-0a59d6522c9f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:17.672719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 30/58] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:17.678052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9241-14e7-420f-bc6a-54d98a017628 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:17.679127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9241-14e7-420f-bc6a-54d98a017628 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:17.679127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9241-14e7-420f-bc6a-54d98a017628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:17.697642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-df0f9241-14e7-420f-bc6a-54d98a017628 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:17.698815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-df0f9241-14e7-420f-bc6a-54d98a017628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:17.720453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:17.771786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 29/59] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:17.797479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9b6d-495e-4dea-ae43-0a59d6522c9f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:17.901217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:17.906254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9b6d-495e-4dea-ae43-0a59d6522c9f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:17.940400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3996c5-0b42-4154-897f-759080f7358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:17.940949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3996c5-0b42-4154-897f-759080f7358c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:17.941248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3996c5-0b42-4154-897f-759080f7358c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:17.941521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3996c5-0b42-4154-897f-759080f7358c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:17.942624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3996c5-0b42-4154-897f-759080f7358c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:17.944083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3996c5-0b42-4154-897f-759080f7358c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:17.946180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 31/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:17.951785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f9241-14e7-420f-bc6a-54d98a017628 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:18.050839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f9241-14e7-420f-bc6a-54d98a017628 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:18.084648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42956e82-b183-41ad-af54-8ccd1c9ea042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:18.085289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956e82-b183-41ad-af54-8ccd1c9ea042 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:18.085616 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956e82-b183-41ad-af54-8ccd1c9ea042 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:18.085895 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956e82-b183-41ad-af54-8ccd1c9ea042 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:18.086601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42956e82-b183-41ad-af54-8ccd1c9ea042 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:18.088055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42956e82-b183-41ad-af54-8ccd1c9ea042 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:18.114916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 30/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:24:17 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:21.274141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b9bea-cc5d-4232-baf7-ee4fd77a1c6a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.274141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b9bea-cc5d-4232-baf7-ee4fd77a1c6a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.274743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b9bea-cc5d-4232-baf7-ee4fd77a1c6a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.275845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 32/62] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:21.280939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d3f9c-90a8-426d-bc5c-577840a1e14e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.281784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d3f9c-90a8-426d-bc5c-577840a1e14e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.282276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d3f9c-90a8-426d-bc5c-577840a1e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.310539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-908d3f9c-90a8-426d-bc5c-577840a1e14e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:21.312086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-908d3f9c-90a8-426d-bc5c-577840a1e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:21.324220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:21.414655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 31/63] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:21.450198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b9bea-cc5d-4232-baf7-ee4fd77a1c6a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:21.498927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869578b-07b7-413f-ac23-aaf1e46673c1 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.499682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869578b-07b7-413f-ac23-aaf1e46673c1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.500329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869578b-07b7-413f-ac23-aaf1e46673c1 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.502841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 32/64] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:21.511820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b24eb-fbd4-457b-be09-8dce93015094 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.512405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b24eb-fbd4-457b-be09-8dce93015094 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.512905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b24eb-fbd4-457b-be09-8dce93015094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.548611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5b8b24eb-fbd4-457b-be09-8dce93015094 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:21.549775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5b8b24eb-fbd4-457b-be09-8dce93015094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:21.673995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b9bea-cc5d-4232-baf7-ee4fd77a1c6a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:21.687160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 33/65] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 13:24:21.729362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db98cb-8fce-46ae-82f3-86cd949c8ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:21.730356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db98cb-8fce-46ae-82f3-86cd949c8ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.730840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db98cb-8fce-46ae-82f3-86cd949c8ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.731339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db98cb-8fce-46ae-82f3-86cd949c8ca1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.741515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7b061-cd41-45b3-9e45-a4ca6e3f0145 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.742731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7b061-cd41-45b3-9e45-a4ca6e3f0145 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.743312 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7b061-cd41-45b3-9e45-a4ca6e3f0145 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.745184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 34/66] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:21.751703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a29f54-da8d-4bcd-a657-d56895b7785f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.752505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a29f54-da8d-4bcd-a657-d56895b7785f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.753272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a29f54-da8d-4bcd-a657-d56895b7785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.754685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db98cb-8fce-46ae-82f3-86cd949c8ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-33db98cb-8fce-46ae-82f3-86cd949c8ca1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:21.760501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:21.761491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:21.762329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 33/67] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:21.762950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db98cb-8fce-46ae-82f3-86cd949c8ca1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:21.780765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-33a29f54-da8d-4bcd-a657-d56895b7785f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:21.782484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-33a29f54-da8d-4bcd-a657-d56895b7785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:21.845082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:21.863186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 35/68] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:21.864047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a29f54-da8d-4bcd-a657-d56895b7785f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:21.869291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:21.914621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db98cb-8fce-46ae-82f3-86cd949c8ca1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:21.963002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87043454-5ebc-49d2-8b7a-2a46ea782010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:21.963652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87043454-5ebc-49d2-8b7a-2a46ea782010 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.964189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87043454-5ebc-49d2-8b7a-2a46ea782010 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.964465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87043454-5ebc-49d2-8b7a-2a46ea782010 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.975240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87043454-5ebc-49d2-8b7a-2a46ea782010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:21.980348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a29f54-da8d-4bcd-a657-d56895b7785f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-87043454-5ebc-49d2-8b7a-2a46ea782010 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:21.980980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:21.982957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 34/69] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:21.984466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e337d9-180c-4458-a1ee-29bd151aed72 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.985791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e337d9-180c-4458-a1ee-29bd151aed72 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.985791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e337d9-180c-4458-a1ee-29bd151aed72 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.985791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 35/70] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:21.986887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced700a-4682-416a-b53e-328c76e2f357 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:21.986887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced700a-4682-416a-b53e-328c76e2f357 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:21.987280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced700a-4682-416a-b53e-328c76e2f357 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:21.989267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 36/71] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:21.989267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced700a-4682-416a-b53e-328c76e2f357 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.023772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50215441-dd42-41e9-a474-85c186078224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.024420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50215441-dd42-41e9-a474-85c186078224 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.024700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50215441-dd42-41e9-a474-85c186078224 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.024970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50215441-dd42-41e9-a474-85c186078224 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.025772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50215441-dd42-41e9-a474-85c186078224 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:22.025963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50215441-dd42-41e9-a474-85c186078224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:22.027385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50215441-dd42-41e9-a474-85c186078224 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:22.036505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 36/72] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:22.037159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50215441-dd42-41e9-a474-85c186078224 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.083728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:22.100451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced700a-4682-416a-b53e-328c76e2f357 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.108501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:22.136706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f571198-5a0c-4f44-beb8-e4f34e5e524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.146445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571198-5a0c-4f44-beb8-e4f34e5e524a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.146721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571198-5a0c-4f44-beb8-e4f34e5e524a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.146987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571198-5a0c-4f44-beb8-e4f34e5e524a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.147909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50215441-dd42-41e9-a474-85c186078224 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.169399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f571198-5a0c-4f44-beb8-e4f34e5e524a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9f571198-5a0c-4f44-beb8-e4f34e5e524a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:22.174132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:22.175027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:22.175782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 37/73] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:24:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:22.176736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90de88-bb82-4bf7-936f-8267bcf3927a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.177072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90de88-bb82-4bf7-936f-8267bcf3927a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.177192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90de88-bb82-4bf7-936f-8267bcf3927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.193671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652f51d-3bca-4ccd-9fea-891893fdfc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.194285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f51d-3bca-4ccd-9fea-891893fdfc03 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.194561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f51d-3bca-4ccd-9fea-891893fdfc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.194979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f51d-3bca-4ccd-9fea-891893fdfc03 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.195663 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652f51d-3bca-4ccd-9fea-891893fdfc03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:22.195848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9b90de88-bb82-4bf7-936f-8267bcf3927a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:22.196014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652f51d-3bca-4ccd-9fea-891893fdfc03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:22.196918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9b90de88-bb82-4bf7-936f-8267bcf3927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:22.197339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652f51d-3bca-4ccd-9fea-891893fdfc03 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:22.207476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 37/74] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:22.208711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a589e7-ddab-46a2-b910-45acbb3c5ae9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.208991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a589e7-ddab-46a2-b910-45acbb3c5ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.209291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a589e7-ddab-46a2-b910-45acbb3c5ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.232671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-58a589e7-ddab-46a2-b910-45acbb3c5ae9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:22.234545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-58a589e7-ddab-46a2-b910-45acbb3c5ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:22.278392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 38/75] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:22.279017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90de88-bb82-4bf7-936f-8267bcf3927a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.312722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 38/76] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:22.312722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a589e7-ddab-46a2-b910-45acbb3c5ae9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.361117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:22.367351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:22.384210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90de88-bb82-4bf7-936f-8267bcf3927a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.409987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a589e7-ddab-46a2-b910-45acbb3c5ae9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.417093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4264f29-a226-4d71-bd4b-7c037ec88398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.417763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4264f29-a226-4d71-bd4b-7c037ec88398 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.418039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4264f29-a226-4d71-bd4b-7c037ec88398 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.418308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4264f29-a226-4d71-bd4b-7c037ec88398 None admin] PATH_INFO: `/v3/users/1d95623897cc4c628d91416fe290bfa2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.419418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4264f29-a226-4d71-bd4b-7c037ec88398 None admin] RBAC: Authorizing `identity:get_user(user_id=1d95623897cc4c628d91416fe290bfa2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:22.420867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4264f29-a226-4d71-bd4b-7c037ec88398 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:22.422620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 39/77] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/users/1d95623897cc4c628d91416fe290bfa2 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:22.423198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4264f29-a226-4d71-bd4b-7c037ec88398 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.447577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.448193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.448494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.448750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.449434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:22.449737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:22.451104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:22.458973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 39/78] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:22.459633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.528741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4264f29-a226-4d71-bd4b-7c037ec88398 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.559597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a3bff-060a-4b8a-8ae1-97f458587667 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.563204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076670e4-5d3c-4240-8ddf-88d2ae9223cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.563829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076670e4-5d3c-4240-8ddf-88d2ae9223cc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.564107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076670e4-5d3c-4240-8ddf-88d2ae9223cc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.564372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076670e4-5d3c-4240-8ddf-88d2ae9223cc None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.575769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:22.578124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076670e4-5d3c-4240-8ddf-88d2ae9223cc None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:22.579742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076670e4-5d3c-4240-8ddf-88d2ae9223cc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:22.581210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 40/79] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:22.581985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076670e4-5d3c-4240-8ddf-88d2ae9223cc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.584393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:22.597601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951809f-e0b6-4356-acd0-318877784c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.598246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951809f-e0b6-4356-acd0-318877784c6c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.598530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951809f-e0b6-4356-acd0-318877784c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.598811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951809f-e0b6-4356-acd0-318877784c6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.611430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951809f-e0b6-4356-acd0-318877784c6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-8951809f-e0b6-4356-acd0-318877784c6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:22.618228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:22.619062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 40/80] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:22.620104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951809f-e0b6-4356-acd0-318877784c6c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.681901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076670e4-5d3c-4240-8ddf-88d2ae9223cc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.714935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541092b8-cf3c-4e9c-944c-510283bb04bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.715741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541092b8-cf3c-4e9c-944c-510283bb04bf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.716175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541092b8-cf3c-4e9c-944c-510283bb04bf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.716452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541092b8-cf3c-4e9c-944c-510283bb04bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.717589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541092b8-cf3c-4e9c-944c-510283bb04bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:22.718629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951809f-e0b6-4356-acd0-318877784c6c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.719040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541092b8-cf3c-4e9c-944c-510283bb04bf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:22.720590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 41/81] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:22.721375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541092b8-cf3c-4e9c-944c-510283bb04bf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.755755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19387-2725-4d9b-9c6f-20b5d93d0fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.756362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19387-2725-4d9b-9c6f-20b5d93d0fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.756664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19387-2725-4d9b-9c6f-20b5d93d0fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.756932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19387-2725-4d9b-9c6f-20b5d93d0fd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.788843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:22.801101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:22.819490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541092b8-cf3c-4e9c-944c-510283bb04bf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.823443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e19387-2725-4d9b-9c6f-20b5d93d0fd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:22.824927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e19387-2725-4d9b-9c6f-20b5d93d0fd1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:22.852740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9e36f-a257-4082-aa56-71884b823c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.853389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e36f-a257-4082-aa56-71884b823c59 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.853663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e36f-a257-4082-aa56-71884b823c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:22.853930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e36f-a257-4082-aa56-71884b823c59 None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:22.855037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9e36f-a257-4082-aa56-71884b823c59 None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:22.856431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9e36f-a257-4082-aa56-71884b823c59 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:22.858264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 42/82] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:22.859013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9e36f-a257-4082-aa56-71884b823c59 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.925071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 41/83] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:22.925797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19387-2725-4d9b-9c6f-20b5d93d0fd1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:22.959970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9e36f-a257-4082-aa56-71884b823c59 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:22.999085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d607cf-00bd-444c-a61e-c5903c8d8325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:22.999686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d607cf-00bd-444c-a61e-c5903c8d8325 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:22.999996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d607cf-00bd-444c-a61e-c5903c8d8325 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.000275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d607cf-00bd-444c-a61e-c5903c8d8325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.000550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:23.008750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:23.021815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19387-2725-4d9b-9c6f-20b5d93d0fd1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:23.055464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:23.056091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:23.056344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.056611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.057279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:23.057578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.058946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.066715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 42/84] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:23.067467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:23.068151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d607cf-00bd-444c-a61e-c5903c8d8325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.069608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d607cf-00bd-444c-a61e-c5903c8d8325 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.165512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bb005-321a-499f-9255-611d6e12fd03 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:23.167147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 43/85] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:23.171902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d607cf-00bd-444c-a61e-c5903c8d8325 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:23.204710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fdc06-ad5f-446e-81bb-64a809f8c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:23.205267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fdc06-ad5f-446e-81bb-64a809f8c3c5 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:23.205546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fdc06-ad5f-446e-81bb-64a809f8c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.205820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fdc06-ad5f-446e-81bb-64a809f8c3c5 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.206470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fdc06-ad5f-446e-81bb-64a809f8c3c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.208625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fdc06-ad5f-446e-81bb-64a809f8c3c5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.212415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:23.220404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:23.233767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 43/86] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:24:23 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:23.238379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fdc06-ad5f-446e-81bb-64a809f8c3c5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:23.303248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d607cf-00bd-444c-a61e-c5903c8d8325 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:23.350056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965fe56-9825-476a-8252-205602202bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:23.351684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965fe56-9825-476a-8252-205602202bfd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:23.351960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965fe56-9825-476a-8252-205602202bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.352239 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965fe56-9825-476a-8252-205602202bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.406690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fdc06-ad5f-446e-81bb-64a809f8c3c5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:23.440779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965fe56-9825-476a-8252-205602202bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.442307 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965fe56-9825-476a-8252-205602202bfd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.450673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011c4a2-4624-4e9f-a58e-a1ce6c5494d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:23.451332 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011c4a2-4624-4e9f-a58e-a1ce6c5494d4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:23.451612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011c4a2-4624-4e9f-a58e-a1ce6c5494d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.451884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011c4a2-4624-4e9f-a58e-a1ce6c5494d4 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.453452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011c4a2-4624-4e9f-a58e-a1ce6c5494d4 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.456080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011c4a2-4624-4e9f-a58e-a1ce6c5494d4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.458719 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 44/87] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:23 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:23.459893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011c4a2-4624-4e9f-a58e-a1ce6c5494d4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:23.491948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:23.546740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:23.595774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 44/88] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:23.596718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965fe56-9825-476a-8252-205602202bfd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:23.607843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011c4a2-4624-4e9f-a58e-a1ce6c5494d4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:23.647041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d35d64-4c88-4cb4-9440-de6d30a3d42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:23.647683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d35d64-4c88-4cb4-9440-de6d30a3d42d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:23.647994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d35d64-4c88-4cb4-9440-de6d30a3d42d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.648267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d35d64-4c88-4cb4-9440-de6d30a3d42d None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.649454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d35d64-4c88-4cb4-9440-de6d30a3d42d None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.651063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d35d64-4c88-4cb4-9440-de6d30a3d42d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.652692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 45/89] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:23 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:23.658734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d35d64-4c88-4cb4-9440-de6d30a3d42d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:23.731655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965fe56-9825-476a-8252-205602202bfd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:23.779192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e6e90-fc2a-4212-87ee-d61ced03bb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:23.780198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6e90-fc2a-4212-87ee-d61ced03bb08 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:23.780675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6e90-fc2a-4212-87ee-d61ced03bb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.781138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6e90-fc2a-4212-87ee-d61ced03bb08 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.782610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e6e90-fc2a-4212-87ee-d61ced03bb08 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.784118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d35d64-4c88-4cb4-9440-de6d30a3d42d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:23.784689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e6e90-fc2a-4212-87ee-d61ced03bb08 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.786609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 45/90] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:23 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:23.787584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e6e90-fc2a-4212-87ee-d61ced03bb08 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:23.797763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:23.821517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:23.828176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:23.829325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:23.829960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.830427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.831383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=8d1b82d0c0604e3a98c466c903e60279, user.id=1d95623897cc4c628d91416fe290bfa2) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:23.831905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.833807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.842176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 46/91] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:23 2022] GET /identity/v3/role_assignments?user.id=1d95623897cc4c628d91416fe290bfa2&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:23.843100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:23.941116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e6e90-fc2a-4212-87ee-d61ced03bb08 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:23.984554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4e61-761b-41c8-b569-c305a740b1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:23.984554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4e61-761b-41c8-b569-c305a740b1de None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:23.984554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4e61-761b-41c8-b569-c305a740b1de None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:23.984554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4e61-761b-41c8-b569-c305a740b1de None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:23.985852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad4e61-761b-41c8-b569-c305a740b1de None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:23.987594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad4e61-761b-41c8-b569-c305a740b1de None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:23.993264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 46/92] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:23 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:23.993264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4e61-761b-41c8-b569-c305a740b1de None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:24.064385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42aeea-8a06-4a7e-b3e9-3328f7382b4e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:24.101980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:24.129980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d34deb-6e95-45c7-b644-379faf99d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:24.130770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d34deb-6e95-45c7-b644-379faf99d7ec None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:24.131154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d34deb-6e95-45c7-b644-379faf99d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:24.131528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d34deb-6e95-45c7-b644-379faf99d7ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:24.152375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4e61-761b-41c8-b569-c305a740b1de None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:24.214453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:24.216153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:24.216883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:24.217649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:24.219230 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] RBAC: Adding query filter params (role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=8d1b82d0c0604e3a98c466c903e60279, user.id=45a4a1663ad84ad1953e68ba332fad3f) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:24.220123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:24.223156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:24.235306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 47/93] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:23 2022] GET /identity/v3/role_assignments?user.id=45a4a1663ad84ad1953e68ba332fad3f&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:24.237188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:24.331864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:24.349466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d34deb-6e95-45c7-b644-379faf99d7ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:24.351180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d34deb-6e95-45c7-b644-379faf99d7ec None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:24.441427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49db15-27c7-440a-a887-498d6ce3966e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:24.507099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:24.508637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:24.509427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:24.510155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:24.511722 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] RBAC: Adding query filter params (role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=8d1b82d0c0604e3a98c466c903e60279, user.id=45a4a1663ad84ad1953e68ba332fad3f) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:24.512504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:24.515524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:24.525905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 48/94] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:24 2022] GET /identity/v3/role_assignments?user.id=45a4a1663ad84ad1953e68ba332fad3f&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:24.553894 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 47/95] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:24.562619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:24.566221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:24.694457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875812bd-b16c-4538-870d-2d25cb3747e4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:24.732193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c47ed-cb57-4cde-8fe0-716438cf102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:24.733485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c47ed-cb57-4cde-8fe0-716438cf102e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:24.733961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c47ed-cb57-4cde-8fe0-716438cf102e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:24.734409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c47ed-cb57-4cde-8fe0-716438cf102e None admin] PATH_INFO: `/v3/projects/7a4d878448904154a14b5d3cfdd7e1b8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:24.736225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c47ed-cb57-4cde-8fe0-716438cf102e None admin] RBAC: Authorizing `identity:get_project(project_id=7a4d878448904154a14b5d3cfdd7e1b8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:24.737929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c47ed-cb57-4cde-8fe0-716438cf102e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:24.740328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 49/96] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:24 2022] GET /identity/v3/projects/7a4d878448904154a14b5d3cfdd7e1b8 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:24.746189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d34deb-6e95-45c7-b644-379faf99d7ec None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:24.866020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:24.872398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d34deb-6e95-45c7-b644-379faf99d7ec None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:24.909881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405d21-7b2c-4669-831d-72785ebeb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:24.910798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405d21-7b2c-4669-831d-72785ebeb98d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:24.911346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405d21-7b2c-4669-831d-72785ebeb98d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:24.911787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405d21-7b2c-4669-831d-72785ebeb98d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:24.912879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405d21-7b2c-4669-831d-72785ebeb98d None admin] RBAC: Adding query filter params (role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=7a4d878448904154a14b5d3cfdd7e1b8, user.id=45a4a1663ad84ad1953e68ba332fad3f) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:24.913510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405d21-7b2c-4669-831d-72785ebeb98d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:24.915655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405d21-7b2c-4669-831d-72785ebeb98d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:24.922200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 48/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:24 2022] GET /identity/v3/role_assignments?user.id=45a4a1663ad84ad1953e68ba332fad3f&scope.project.id=7a4d878448904154a14b5d3cfdd7e1b8&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:26.012793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7ccf-f9ce-4b60-9188-3f9b2de60440 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:26.013369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7ccf-f9ce-4b60-9188-3f9b2de60440 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:26.013369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7ccf-f9ce-4b60-9188-3f9b2de60440 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:26.014981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 50/98] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:26.020377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d822-e5d1-487d-9c64-feffc35fd3b5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:26.020810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d822-e5d1-487d-9c64-feffc35fd3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:26.021003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d822-e5d1-487d-9c64-feffc35fd3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:26.040542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1549d822-e5d1-487d-9c64-feffc35fd3b5 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:26.041761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1549d822-e5d1-487d-9c64-feffc35fd3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:26.139534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 49/99] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:26.162094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef7ccf-f9ce-4b60-9188-3f9b2de60440 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:26.236659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:26.278293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef7ccf-f9ce-4b60-9188-3f9b2de60440 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:26.326052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf3bb-cf12-4ddb-9370-6a3f6412723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:26.326749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf3bb-cf12-4ddb-9370-6a3f6412723a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:26.327032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf3bb-cf12-4ddb-9370-6a3f6412723a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:26.327305 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf3bb-cf12-4ddb-9370-6a3f6412723a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:26.328398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf3bb-cf12-4ddb-9370-6a3f6412723a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:26.329886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf3bb-cf12-4ddb-9370-6a3f6412723a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:26.331447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 51/100] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:26.338040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549d822-e5d1-487d-9c64-feffc35fd3b5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:26.455342 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:26.465322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549d822-e5d1-487d-9c64-feffc35fd3b5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:26.500766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49e374-02e0-43f7-a23a-0c69666e31f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:26.501472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49e374-02e0-43f7-a23a-0c69666e31f4 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:26.501753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49e374-02e0-43f7-a23a-0c69666e31f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:26.502023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49e374-02e0-43f7-a23a-0c69666e31f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:26.502683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49e374-02e0-43f7-a23a-0c69666e31f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:26.504144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49e374-02e0-43f7-a23a-0c69666e31f4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:26.543886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 50/101] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:24:26 2022] POST /identity/v3/projects => generated 302 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:26.555282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b2761-e826-4105-bcac-e6dfd9c70b10 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:26.555574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b2761-e826-4105-bcac-e6dfd9c70b10 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:26.555886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b2761-e826-4105-bcac-e6dfd9c70b10 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:26.557498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 52/102] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:26.569613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590699-5d1e-485c-a09c-829b862896fb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:26.569910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590699-5d1e-485c-a09c-829b862896fb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:26.570157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590699-5d1e-485c-a09c-829b862896fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:26.599213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d5590699-5d1e-485c-a09c-829b862896fb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:26.600441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d5590699-5d1e-485c-a09c-829b862896fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:26.704001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 51/103] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:26.748206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b2761-e826-4105-bcac-e6dfd9c70b10 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:26.893185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b2761-e826-4105-bcac-e6dfd9c70b10 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:26.937435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe20b6-22ce-48ca-b880-1a01ff3044ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:26.938525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe20b6-22ce-48ca-b880-1a01ff3044ff None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:26.938805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe20b6-22ce-48ca-b880-1a01ff3044ff None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:26.939164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe20b6-22ce-48ca-b880-1a01ff3044ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:26.948772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:26.953036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe20b6-22ce-48ca-b880-1a01ff3044ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-85fe20b6-22ce-48ca-b880-1a01ff3044ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:26.960132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:26.961678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:26.962445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 53/104] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:26.968375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5590699-5d1e-485c-a09c-829b862896fb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:27.082770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:27.095082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bbd32-24f0-4d11-8192-736eb611cdf8 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.095599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bbd32-24f0-4d11-8192-736eb611cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.095599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bbd32-24f0-4d11-8192-736eb611cdf8 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.097017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 54/105] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:27.110686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f78559-0b37-46f6-a95b-46e03b77fb33 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.110996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f78559-0b37-46f6-a95b-46e03b77fb33 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.111232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f78559-0b37-46f6-a95b-46e03b77fb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.137104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-52f78559-0b37-46f6-a95b-46e03b77fb33 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:27.138480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-52f78559-0b37-46f6-a95b-46e03b77fb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:27.139886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5590699-5d1e-485c-a09c-829b862896fb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:27.199422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:27.200329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.200798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.201394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.202086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:27.202393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:27.206877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:27.222996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 55/106] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:27.223371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 52/107] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:27.234302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f78559-0b37-46f6-a95b-46e03b77fb33 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:27.254448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:27.417059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b98caf-ebc7-49de-a3af-e645e641111d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:27.455961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f78559-0b37-46f6-a95b-46e03b77fb33 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:27.465414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36647b74-56af-41da-b29c-a82480d429c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:27.466074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36647b74-56af-41da-b29c-a82480d429c1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.466348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36647b74-56af-41da-b29c-a82480d429c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.466651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36647b74-56af-41da-b29c-a82480d429c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.474991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:27.494134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:27.505901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36647b74-56af-41da-b29c-a82480d429c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:27.514686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff6486-bb02-4a81-af9b-6d20478a9a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:27.515655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff6486-bb02-4a81-af9b-6d20478a9a9b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.516134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff6486-bb02-4a81-af9b-6d20478a9a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.516603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff6486-bb02-4a81-af9b-6d20478a9a9b None admin] PATH_INFO: `/v3/users/e24fdc860b5246a6a2242933f5e96c75` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-36647b74-56af-41da-b29c-a82480d429c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:27.518474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:27.519337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:27.520096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:27.524760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 53/108] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:27.531709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b42ec-4e7e-450e-bebf-32c00d62f31d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.534780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b42ec-4e7e-450e-bebf-32c00d62f31d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.537129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b42ec-4e7e-450e-bebf-32c00d62f31d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.538908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 54/109] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:27.545002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f9bb-5e18-4f41-8b1d-246d06eb5e0b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.547085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f9bb-5e18-4f41-8b1d-246d06eb5e0b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.547593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f9bb-5e18-4f41-8b1d-246d06eb5e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.557134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff6486-bb02-4a81-af9b-6d20478a9a9b None admin] RBAC: Authorizing `identity:get_user(user_id=e24fdc860b5246a6a2242933f5e96c75)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:27.559284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff6486-bb02-4a81-af9b-6d20478a9a9b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:27.562403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 56/110] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:27 2022] GET /identity/v3/users/e24fdc860b5246a6a2242933f5e96c75 => generated 289 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:27.563540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff6486-bb02-4a81-af9b-6d20478a9a9b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:27.586825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b7c2f9bb-5e18-4f41-8b1d-246d06eb5e0b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:27.588049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b7c2f9bb-5e18-4f41-8b1d-246d06eb5e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:27.719291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 55/111] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:27.720462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2f9bb-5e18-4f41-8b1d-246d06eb5e0b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:27.777283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff6486-bb02-4a81-af9b-6d20478a9a9b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:27.807528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840be4ee-5e99-46c5-97f2-232c3fd91fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:27.808201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840be4ee-5e99-46c5-97f2-232c3fd91fbf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.808483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840be4ee-5e99-46c5-97f2-232c3fd91fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.808756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840be4ee-5e99-46c5-97f2-232c3fd91fbf None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.809459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840be4ee-5e99-46c5-97f2-232c3fd91fbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:27.809764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840be4ee-5e99-46c5-97f2-232c3fd91fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:27.811168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840be4ee-5e99-46c5-97f2-232c3fd91fbf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:27.820829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 57/112] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:27.826630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59f015-ea42-4359-aa05-4c0b5c0c9c0d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.826914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59f015-ea42-4359-aa05-4c0b5c0c9c0d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.827190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59f015-ea42-4359-aa05-4c0b5c0c9c0d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.828607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 58/113] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:27.830072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59f015-ea42-4359-aa05-4c0b5c0c9c0d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:27.868547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2f9bb-5e18-4f41-8b1d-246d06eb5e0b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:27.883659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:27.899339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:27.914044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99332aa-8a86-477f-81fe-0c700dacf539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:27.915452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99332aa-8a86-477f-81fe-0c700dacf539 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.915940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99332aa-8a86-477f-81fe-0c700dacf539 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.916413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99332aa-8a86-477f-81fe-0c700dacf539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.945084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59f015-ea42-4359-aa05-4c0b5c0c9c0d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:27.984513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325bb5f-3ff3-409c-81b0-9097dad008da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:27.985080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325bb5f-3ff3-409c-81b0-9097dad008da None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.985395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325bb5f-3ff3-409c-81b0-9097dad008da None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.985670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325bb5f-3ff3-409c-81b0-9097dad008da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.998654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99332aa-8a86-477f-81fe-0c700dacf539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:27.999564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325bb5f-3ff3-409c-81b0-9097dad008da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:28.000807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99332aa-8a86-477f-81fe-0c700dacf539 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4325bb5f-3ff3-409c-81b0-9097dad008da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:28.006135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:28.007135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:28.007903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 59/114] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:28.008528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325bb5f-3ff3-409c-81b0-9097dad008da None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.126662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 56/115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:28.128228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325bb5f-3ff3-409c-81b0-9097dad008da None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.129603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463cfda-90a8-4831-89a1-729b7aabd700 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.130333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463cfda-90a8-4831-89a1-729b7aabd700 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.130906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463cfda-90a8-4831-89a1-729b7aabd700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.138145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:28.153441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e463cfda-90a8-4831-89a1-729b7aabd700 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:28.155070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e463cfda-90a8-4831-89a1-729b7aabd700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:28.164861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b1767-e88f-4634-8487-77f719f91193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.165522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b1767-e88f-4634-8487-77f719f91193 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.165796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b1767-e88f-4634-8487-77f719f91193 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.166084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b1767-e88f-4634-8487-77f719f91193 None admin] PATH_INFO: `/v3/users/1d95623897cc4c628d91416fe290bfa2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.167251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b1767-e88f-4634-8487-77f719f91193 None admin] RBAC: Authorizing `identity:get_user(user_id=1d95623897cc4c628d91416fe290bfa2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:28.168629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b1767-e88f-4634-8487-77f719f91193 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.170234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 60/116] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/users/1d95623897cc4c628d91416fe290bfa2 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:28.170986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b1767-e88f-4634-8487-77f719f91193 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.179881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:28.234481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 57/117] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:28.235398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463cfda-90a8-4831-89a1-729b7aabd700 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.286538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b1767-e88f-4634-8487-77f719f91193 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.322827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a84a8a-c702-4f4a-ba64-7a84f1c8902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.323700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a84a8a-c702-4f4a-ba64-7a84f1c8902c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.324164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a84a8a-c702-4f4a-ba64-7a84f1c8902c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.324614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a84a8a-c702-4f4a-ba64-7a84f1c8902c None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.325512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a84a8a-c702-4f4a-ba64-7a84f1c8902c None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:28.325821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a84a8a-c702-4f4a-ba64-7a84f1c8902c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:28.327261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a84a8a-c702-4f4a-ba64-7a84f1c8902c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.336350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 61/118] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:28.337626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcbbe0-d1ab-4970-a166-1d63917183a7 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.337911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcbbe0-d1ab-4970-a166-1d63917183a7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.338185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcbbe0-d1ab-4970-a166-1d63917183a7 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.339410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 62/119] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:28.340113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcbbe0-d1ab-4970-a166-1d63917183a7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.357750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463cfda-90a8-4831-89a1-729b7aabd700 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.384810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:28.396511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626eafb-c8f0-4cd9-b413-89490a812b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.396882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626eafb-c8f0-4cd9-b413-89490a812b85 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.397158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626eafb-c8f0-4cd9-b413-89490a812b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.397487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626eafb-c8f0-4cd9-b413-89490a812b85 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.399031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626eafb-c8f0-4cd9-b413-89490a812b85 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:28.401155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626eafb-c8f0-4cd9-b413-89490a812b85 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.407509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 58/120] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:28.407509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626eafb-c8f0-4cd9-b413-89490a812b85 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.435256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:28.467932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcbbe0-d1ab-4970-a166-1d63917183a7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.512004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27790-c35a-4cb7-ba72-23b19949a23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.512552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27790-c35a-4cb7-ba72-23b19949a23f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.512881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27790-c35a-4cb7-ba72-23b19949a23f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.513158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27790-c35a-4cb7-ba72-23b19949a23f None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.514306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db27790-c35a-4cb7-ba72-23b19949a23f None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:28.515827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db27790-c35a-4cb7-ba72-23b19949a23f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.517788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 63/121] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:28.518409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27790-c35a-4cb7-ba72-23b19949a23f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.528201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626eafb-c8f0-4cd9-b413-89490a812b85 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.575677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071b4ca-868c-47bd-9127-14b80198e1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.576351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b4ca-868c-47bd-9127-14b80198e1cb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.576604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b4ca-868c-47bd-9127-14b80198e1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.576874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b4ca-868c-47bd-9127-14b80198e1cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.587870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071b4ca-868c-47bd-9127-14b80198e1cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b071b4ca-868c-47bd-9127-14b80198e1cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:28.593842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:28.595491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 59/122] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:28.596913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0411b5f-bd73-4e8e-b671-aa73c8710d7d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.599988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0411b5f-bd73-4e8e-b671-aa73c8710d7d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.599988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0411b5f-bd73-4e8e-b671-aa73c8710d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.618498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b0411b5f-bd73-4e8e-b671-aa73c8710d7d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:28.620086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b0411b5f-bd73-4e8e-b671-aa73c8710d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:28.639332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27790-c35a-4cb7-ba72-23b19949a23f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.646769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:28.676719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63d071-7097-468a-9d17-fa8aebd00ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.677476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63d071-7097-468a-9d17-fa8aebd00ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.677755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63d071-7097-468a-9d17-fa8aebd00ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.678024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63d071-7097-468a-9d17-fa8aebd00ae8 None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.680320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63d071-7097-468a-9d17-fa8aebd00ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:28.681832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63d071-7097-468a-9d17-fa8aebd00ae8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.683421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 64/123] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:28.684188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63d071-7097-468a-9d17-fa8aebd00ae8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.699452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 60/124] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:28.700479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0411b5f-bd73-4e8e-b671-aa73c8710d7d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.718213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:28.814061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0411b5f-bd73-4e8e-b671-aa73c8710d7d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.832952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63d071-7097-468a-9d17-fa8aebd00ae8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.849245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a3ece-d655-4461-9fd9-f1fc1bd3c87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.849923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a3ece-d655-4461-9fd9-f1fc1bd3c87e None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.850198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a3ece-d655-4461-9fd9-f1fc1bd3c87e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.850467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a3ece-d655-4461-9fd9-f1fc1bd3c87e None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279/users/1d95623897cc4c628d91416fe290bfa2/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.852423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a3ece-d655-4461-9fd9-f1fc1bd3c87e None admin] RBAC: Authorizing `identity:create_grant(project_id=8d1b82d0c0604e3a98c466c903e60279, user_id=1d95623897cc4c628d91416fe290bfa2, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:28.853893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a3ece-d655-4461-9fd9-f1fc1bd3c87e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.869352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 61/125] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:28 2022] PUT /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279/users/1d95623897cc4c628d91416fe290bfa2/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:28.870559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a3ece-d655-4461-9fd9-f1fc1bd3c87e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.895496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.896126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.896452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.896774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.897581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] RBAC: Adding query filter params (role.id=9ee2cd43468941e28f08b0be54868574, scope.project.id=8d1b82d0c0604e3a98c466c903e60279, user.id=e24fdc860b5246a6a2242933f5e96c75) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:28.897968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:28.899761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.906478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 65/126] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/role_assignments?user.id=e24fdc860b5246a6a2242933f5e96c75&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=9ee2cd43468941e28f08b0be54868574 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:28.918552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:29.019089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:29.019691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:29.101380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ca1ff-c3ef-4a97-8225-a0a19c8f33f3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:29.107710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a3ece-d655-4461-9fd9-f1fc1bd3c87e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:29.150895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9770f3-3041-4f6b-8998-9f96cf917f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:29.151541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9770f3-3041-4f6b-8998-9f96cf917f5a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.151827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9770f3-3041-4f6b-8998-9f96cf917f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.152100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9770f3-3041-4f6b-8998-9f96cf917f5a None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.154071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9770f3-3041-4f6b-8998-9f96cf917f5a None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:29.154760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:29.155539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9770f3-3041-4f6b-8998-9f96cf917f5a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:29.155969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.156404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.156810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.157299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 66/127] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:29.157804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:29.158077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9770f3-3041-4f6b-8998-9f96cf917f5a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:29.158403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:29.160026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:29.171410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 62/128] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:29.172449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:29.288109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008494b4-7c6b-47d3-bd24-682966bebf53 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:29.308113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9770f3-3041-4f6b-8998-9f96cf917f5a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:29.319592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:29.355293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0814ba8-a6f3-4649-b6a6-ffa6556990e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:29.355493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b929-4492-4698-be50-82e86984442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:29.355896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0814ba8-a6f3-4649-b6a6-ffa6556990e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.356184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0814ba8-a6f3-4649-b6a6-ffa6556990e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.356436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0814ba8-a6f3-4649-b6a6-ffa6556990e7 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279/users/45a4a1663ad84ad1953e68ba332fad3f/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.356921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b929-4492-4698-be50-82e86984442b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.357614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b929-4492-4698-be50-82e86984442b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.358036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b929-4492-4698-be50-82e86984442b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.358764 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0814ba8-a6f3-4649-b6a6-ffa6556990e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d1b82d0c0604e3a98c466c903e60279, user_id=45a4a1663ad84ad1953e68ba332fad3f, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:29.360677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0814ba8-a6f3-4649-b6a6-ffa6556990e7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:29.363620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:29.377089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394b929-4492-4698-be50-82e86984442b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1394b929-4492-4698-be50-82e86984442b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:29.383005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:29.384077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:29.385117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:29.386714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 67/129] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:29.387446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b929-4492-4698-be50-82e86984442b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:29.387541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 63/130] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:29 2022] PUT /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279/users/45a4a1663ad84ad1953e68ba332fad3f/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:29.389703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16792-48d5-4dbe-8df0-360400f078ab None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.390206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16792-48d5-4dbe-8df0-360400f078ab None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.390680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16792-48d5-4dbe-8df0-360400f078ab None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.401355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 64/131] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:29.402983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16792-48d5-4dbe-8df0-360400f078ab None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:29.541178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16792-48d5-4dbe-8df0-360400f078ab None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:29.596232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b929-4492-4698-be50-82e86984442b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:29.643672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622267e6-0083-401c-86c2-480f2f833f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:29.645000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622267e6-0083-401c-86c2-480f2f833f1e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.646787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622267e6-0083-401c-86c2-480f2f833f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.647578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622267e6-0083-401c-86c2-480f2f833f1e None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.649884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622267e6-0083-401c-86c2-480f2f833f1e None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:29.650262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:29.650940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.651225 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.651498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.652267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:29.652504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:29.653868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:29.654384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622267e6-0083-401c-86c2-480f2f833f1e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:29.656598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 68/132] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:29 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:29.658528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c60a4e-b2b4-4821-a58e-b5f2d10265fd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.659020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c60a4e-b2b4-4821-a58e-b5f2d10265fd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.659522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c60a4e-b2b4-4821-a58e-b5f2d10265fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.664870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 65/133] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:29.665766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:29.685362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-41c60a4e-b2b4-4821-a58e-b5f2d10265fd None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:29.686957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-41c60a4e-b2b4-4821-a58e-b5f2d10265fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:29.717163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:29.741601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:29.764250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 69/134] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:29.765427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c60a4e-b2b4-4821-a58e-b5f2d10265fd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:29.818753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ab8bc-dc9d-4d5c-a79b-75411ac8f287 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:29.885840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd7c60-0bf7-4858-8e3d-98fc55b052b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:29.886494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd7c60-0bf7-4858-8e3d-98fc55b052b8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:29.886777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd7c60-0bf7-4858-8e3d-98fc55b052b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:29.887051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd7c60-0bf7-4858-8e3d-98fc55b052b8 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:29.889128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd7c60-0bf7-4858-8e3d-98fc55b052b8 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:29.890612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd7c60-0bf7-4858-8e3d-98fc55b052b8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:29.892278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 66/135] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:29 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:29.893092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd7c60-0bf7-4858-8e3d-98fc55b052b8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:29.975472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c60a4e-b2b4-4821-a58e-b5f2d10265fd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:30.023367 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd7c60-0bf7-4858-8e3d-98fc55b052b8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:30.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc7f07-d53f-422f-831e-657b5da616d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:30.043813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc7f07-d53f-422f-831e-657b5da616d9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:30.044091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc7f07-d53f-422f-831e-657b5da616d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:30.044369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc7f07-d53f-422f-831e-657b5da616d9 None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:30.045549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc7f07-d53f-422f-831e-657b5da616d9 None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:30.047004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc7f07-d53f-422f-831e-657b5da616d9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:30.048109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:30.048781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 70/136] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:29 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:30.049366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc7f07-d53f-422f-831e-657b5da616d9 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:30.060037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b3f20-5096-4a73-bdc6-77038101039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:30.060682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b3f20-5096-4a73-bdc6-77038101039c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:30.060968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b3f20-5096-4a73-bdc6-77038101039c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:30.061267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b3f20-5096-4a73-bdc6-77038101039c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:30.062584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b3f20-5096-4a73-bdc6-77038101039c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:30.064932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b3f20-5096-4a73-bdc6-77038101039c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:30.066856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 67/137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:30.068028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b3f20-5096-4a73-bdc6-77038101039c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:30.092207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:30.195138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc7f07-d53f-422f-831e-657b5da616d9 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:30.268829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cbf6e-6465-4879-b67c-fb34e3ae115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:30.269806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbf6e-6465-4879-b67c-fb34e3ae115f None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:30.270280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbf6e-6465-4879-b67c-fb34e3ae115f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:30.276266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbf6e-6465-4879-b67c-fb34e3ae115f None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279/users/45a4a1663ad84ad1953e68ba332fad3f/roles/d0ccb065ab134c66b5033b67d0bb86b6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:30.280448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cbf6e-6465-4879-b67c-fb34e3ae115f None admin] RBAC: Authorizing `identity:create_grant(project_id=8d1b82d0c0604e3a98c466c903e60279, user_id=45a4a1663ad84ad1953e68ba332fad3f, role_id=d0ccb065ab134c66b5033b67d0bb86b6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:30.283314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cbf6e-6465-4879-b67c-fb34e3ae115f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:30.301314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b3f20-5096-4a73-bdc6-77038101039c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:30.319859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 71/138] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:30 2022] PUT /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279/users/45a4a1663ad84ad1953e68ba332fad3f/roles/d0ccb065ab134c66b5033b67d0bb86b6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:30.323017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cbf6e-6465-4879-b67c-fb34e3ae115f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:30.354813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cccecb-480d-4d7b-aee1-a57cdbbddda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:30.356016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cccecb-480d-4d7b-aee1-a57cdbbddda3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:30.356519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cccecb-480d-4d7b-aee1-a57cdbbddda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:30.356980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cccecb-480d-4d7b-aee1-a57cdbbddda3 None admin] PATH_INFO: `/v3/projects/7a4d878448904154a14b5d3cfdd7e1b8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:30.359006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cccecb-480d-4d7b-aee1-a57cdbbddda3 None admin] RBAC: Authorizing `identity:get_project(project_id=7a4d878448904154a14b5d3cfdd7e1b8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:30.360850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cccecb-480d-4d7b-aee1-a57cdbbddda3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:30.363030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 68/139] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:30 2022] GET /identity/v3/projects/7a4d878448904154a14b5d3cfdd7e1b8 => generated 312 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:30.369638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cccecb-480d-4d7b-aee1-a57cdbbddda3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:30.482756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:30.527641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cccecb-480d-4d7b-aee1-a57cdbbddda3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:30.547722 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:30.565292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0168bf-9372-4a1d-b4b5-f97ffd4aa5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:30.566075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0168bf-9372-4a1d-b4b5-f97ffd4aa5ae None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:30.566502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0168bf-9372-4a1d-b4b5-f97ffd4aa5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:30.566855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0168bf-9372-4a1d-b4b5-f97ffd4aa5ae None admin] PATH_INFO: `/v3/projects/7a4d878448904154a14b5d3cfdd7e1b8/users/45a4a1663ad84ad1953e68ba332fad3f/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:30.569448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0168bf-9372-4a1d-b4b5-f97ffd4aa5ae None admin] RBAC: Authorizing `identity:create_grant(project_id=7a4d878448904154a14b5d3cfdd7e1b8, user_id=45a4a1663ad84ad1953e68ba332fad3f, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:30.571582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0168bf-9372-4a1d-b4b5-f97ffd4aa5ae None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:30.586017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cbf6e-6465-4879-b67c-fb34e3ae115f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:30.594070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 69/140] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:30 2022] PUT /identity/v3/projects/7a4d878448904154a14b5d3cfdd7e1b8/users/45a4a1663ad84ad1953e68ba332fad3f/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:30.630653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1501f2-8704-476c-8896-0d1c790a4885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:30.631283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1501f2-8704-476c-8896-0d1c790a4885 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:30.631564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1501f2-8704-476c-8896-0d1c790a4885 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:30.631852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1501f2-8704-476c-8896-0d1c790a4885 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:30.633078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1501f2-8704-476c-8896-0d1c790a4885 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:30.634590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1501f2-8704-476c-8896-0d1c790a4885 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:30.671930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 72/141] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:24:30 2022] POST /identity/v3/users => generated 283 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:31.673384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c5f6d-c31d-4fef-80e8-9c19d100bef4 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:31.673915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c5f6d-c31d-4fef-80e8-9c19d100bef4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:31.673915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c5f6d-c31d-4fef-80e8-9c19d100bef4 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:31.675561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 70/142] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:31.681420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603cd42-894a-460e-9f25-1c3691c3d4ec None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:31.682333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603cd42-894a-460e-9f25-1c3691c3d4ec None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:31.682812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603cd42-894a-460e-9f25-1c3691c3d4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:31.705186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a603cd42-894a-460e-9f25-1c3691c3d4ec None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:31.706803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a603cd42-894a-460e-9f25-1c3691c3d4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:31.712269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4015-66ce-48d4-b72c-db72d149aba5 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:31.712562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4015-66ce-48d4-b72c-db72d149aba5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:31.712843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4015-66ce-48d4-b72c-db72d149aba5 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:31.714242 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 71/143] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:31.726820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3399-91eb-4335-8c27-6b590969ae58 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:31.727020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3399-91eb-4335-8c27-6b590969ae58 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:31.727240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3399-91eb-4335-8c27-6b590969ae58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:31.757319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1fab3399-91eb-4335-8c27-6b590969ae58 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:31.758541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1fab3399-91eb-4335-8c27-6b590969ae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:31.786005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 73/144] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:31.816925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603cd42-894a-460e-9f25-1c3691c3d4ec None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:31.871503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 72/145] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:31.904595 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:31.906501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab3399-91eb-4335-8c27-6b590969ae58 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:31.926869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603cd42-894a-460e-9f25-1c3691c3d4ec None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:31.963409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18143b-5c9a-45fa-99e7-fb3b0d606d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:31.964420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18143b-5c9a-45fa-99e7-fb3b0d606d2f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:31.964907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18143b-5c9a-45fa-99e7-fb3b0d606d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:31.965429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18143b-5c9a-45fa-99e7-fb3b0d606d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:31.979810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18143b-5c9a-45fa-99e7-fb3b0d606d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9e18143b-5c9a-45fa-99e7-fb3b0d606d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:31.986231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:31.987170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:31.987937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:31.990795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 74/146] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:31.997871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18143b-5c9a-45fa-99e7-fb3b0d606d2f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:32.036617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:32.111975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18143b-5c9a-45fa-99e7-fb3b0d606d2f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:32.152908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:32.154148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:32.154638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:32.155098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:32.155967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:32.156465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:32.158555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:32.165571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:32.171049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 75/147] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:32.182252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:32.203515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab3399-91eb-4335-8c27-6b590969ae58 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:32.277015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b55c69-fb19-4dfe-a879-1204f2ae5503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:32.277713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b55c69-fb19-4dfe-a879-1204f2ae5503 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:32.277988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b55c69-fb19-4dfe-a879-1204f2ae5503 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:32.278257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b55c69-fb19-4dfe-a879-1204f2ae5503 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:32.293726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b55c69-fb19-4dfe-a879-1204f2ae5503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-61b55c69-fb19-4dfe-a879-1204f2ae5503 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:32.299223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:32.300045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:32.301039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 73/148] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:32.310397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b55c69-fb19-4dfe-a879-1204f2ae5503 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:32.451832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b55c69-fb19-4dfe-a879-1204f2ae5503 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:32.465421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:32.486127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107ebbd-d6a5-4fa7-b470-b81cba25fac5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:32.500694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:32.501331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:32.501606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:32.501867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:32.503101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:32.503398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:32.504787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:32.519197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 74/149] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:32.525674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:32.586062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c70aa-71c4-4d16-bd57-589844480a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:32.586823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c70aa-71c4-4d16-bd57-589844480a2c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:32.587099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c70aa-71c4-4d16-bd57-589844480a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:32.587367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c70aa-71c4-4d16-bd57-589844480a2c None admin] PATH_INFO: `/v3/users/1d95623897cc4c628d91416fe290bfa2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:32.588568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c70aa-71c4-4d16-bd57-589844480a2c None admin] RBAC: Authorizing `identity:get_user(user_id=1d95623897cc4c628d91416fe290bfa2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:32.590171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c70aa-71c4-4d16-bd57-589844480a2c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:32.592016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 76/150] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:32 2022] GET /identity/v3/users/1d95623897cc4c628d91416fe290bfa2 => generated 247 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:32.622716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c70aa-71c4-4d16-bd57-589844480a2c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:32.648553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72987721-dd50-4b63-90b0-a46f2710ee65 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:32.687250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf241d4-0aed-4b3a-ae70-1c5c90f3aa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:32.687910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf241d4-0aed-4b3a-ae70-1c5c90f3aa3a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:32.688155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf241d4-0aed-4b3a-ae70-1c5c90f3aa3a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:32.688422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf241d4-0aed-4b3a-ae70-1c5c90f3aa3a None admin] PATH_INFO: `/v3/users/e24fdc860b5246a6a2242933f5e96c75` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:32.689603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf241d4-0aed-4b3a-ae70-1c5c90f3aa3a None admin] RBAC: Authorizing `identity:get_user(user_id=e24fdc860b5246a6a2242933f5e96c75)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:32.691509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf241d4-0aed-4b3a-ae70-1c5c90f3aa3a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:32.693453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 75/151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:32 2022] GET /identity/v3/users/e24fdc860b5246a6a2242933f5e96c75 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:32.700006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf241d4-0aed-4b3a-ae70-1c5c90f3aa3a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:32.805723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:32.816654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf241d4-0aed-4b3a-ae70-1c5c90f3aa3a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:32.860040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac86e19-34c2-417e-87e8-e7c7efe62b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:32.860772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86e19-34c2-417e-87e8-e7c7efe62b31 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:32.861049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86e19-34c2-417e-87e8-e7c7efe62b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:32.861340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86e19-34c2-417e-87e8-e7c7efe62b31 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:32.862665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac86e19-34c2-417e-87e8-e7c7efe62b31 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:32.864095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac86e19-34c2-417e-87e8-e7c7efe62b31 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:32.866441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 76/152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:32 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:32.867352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44afdf8-a03b-461c-9dba-7c3c65363412 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:32.867629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44afdf8-a03b-461c-9dba-7c3c65363412 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:32.867891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44afdf8-a03b-461c-9dba-7c3c65363412 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:32.872870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 77/153] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:32.877017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93652-046b-4cb4-8672-8a5506bcc5c6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:32.877322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93652-046b-4cb4-8672-8a5506bcc5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:32.877580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93652-046b-4cb4-8672-8a5506bcc5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:32.897608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:32.898892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e0d93652-046b-4cb4-8672-8a5506bcc5c6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:32.899873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e0d93652-046b-4cb4-8672-8a5506bcc5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:32.976986 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 78/154] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:32.977845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d93652-046b-4cb4-8672-8a5506bcc5c6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:33.002435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c70aa-71c4-4d16-bd57-589844480a2c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:33.058290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797f11-f2a0-4a75-a21a-93c62151835b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:33.058828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797f11-f2a0-4a75-a21a-93c62151835b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:33.059103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797f11-f2a0-4a75-a21a-93c62151835b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:33.059370 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e797f11-f2a0-4a75-a21a-93c62151835b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:33.121592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:33.142802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e797f11-f2a0-4a75-a21a-93c62151835b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:33.144292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e797f11-f2a0-4a75-a21a-93c62151835b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:33.256269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:33.321339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 77/155] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:24:33.321339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797f11-f2a0-4a75-a21a-93c62151835b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:33.352671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d93652-046b-4cb4-8672-8a5506bcc5c6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:33.493752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b1b9-c596-40d1-b8ae-3e5930a1d922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:33.494936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b1b9-c596-40d1-b8ae-3e5930a1d922 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:33.495592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b1b9-c596-40d1-b8ae-3e5930a1d922 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:33.496219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b1b9-c596-40d1-b8ae-3e5930a1d922 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279/users/e24fdc860b5246a6a2242933f5e96c75/roles/9ee2cd43468941e28f08b0be54868574` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:33.503619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5b1b9-c596-40d1-b8ae-3e5930a1d922 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d1b82d0c0604e3a98c466c903e60279, user_id=e24fdc860b5246a6a2242933f5e96c75, role_id=9ee2cd43468941e28f08b0be54868574)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:33.506294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5b1b9-c596-40d1-b8ae-3e5930a1d922 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:33.547081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 79/156] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:32 2022] PUT /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279/users/e24fdc860b5246a6a2242933f5e96c75/roles/9ee2cd43468941e28f08b0be54868574 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:33.548878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b1b9-c596-40d1-b8ae-3e5930a1d922 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:33.561704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e797f11-f2a0-4a75-a21a-93c62151835b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:33.640282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22cca8-56a1-446c-860e-63b46e1cf4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:33.641511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22cca8-56a1-446c-860e-63b46e1cf4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:33.642004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22cca8-56a1-446c-860e-63b46e1cf4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:33.642461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22cca8-56a1-446c-860e-63b46e1cf4b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:33.665409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22cca8-56a1-446c-860e-63b46e1cf4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9a22cca8-56a1-446c-860e-63b46e1cf4b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:33.671553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:33.672721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:33.673604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:33.676922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 78/157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:33.682940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22cca8-56a1-446c-860e-63b46e1cf4b6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:33.729514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:33.749629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:33.755776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b1b9-c596-40d1-b8ae-3e5930a1d922 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:33.811047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd28dfc-c9b7-42a3-b95e-796c873b015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:33.811726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd28dfc-c9b7-42a3-b95e-796c873b015a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:33.811998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd28dfc-c9b7-42a3-b95e-796c873b015a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:33.813200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd28dfc-c9b7-42a3-b95e-796c873b015a None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:33.815290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd28dfc-c9b7-42a3-b95e-796c873b015a None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:33.816960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd28dfc-c9b7-42a3-b95e-796c873b015a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:33.819699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 80/158] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:33 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:33.831484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd28dfc-c9b7-42a3-b95e-796c873b015a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:33.883230 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22cca8-56a1-446c-860e-63b46e1cf4b6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:33.955088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:33.957079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:33.957790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:33.958264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:33.959162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:33.959659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:33.962000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:33.979931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 79/159] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:33.986081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:34.025589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd28dfc-c9b7-42a3-b95e-796c873b015a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:34.091889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:34.092562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:34.092956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:34.093241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:34.094007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=8d1b82d0c0604e3a98c466c903e60279, user.id=1d95623897cc4c628d91416fe290bfa2) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:34.094602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:34.096301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:34.106759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 81/160] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:33 2022] GET /identity/v3/role_assignments?user.id=1d95623897cc4c628d91416fe290bfa2&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:34.152210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:34.170543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe8971-08ca-44f6-a791-d528bd0d500d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:34.219729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f86e0e-4793-4fba-9d5f-a631d3429dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:34.220761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f86e0e-4793-4fba-9d5f-a631d3429dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:34.221286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f86e0e-4793-4fba-9d5f-a631d3429dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:34.221777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f86e0e-4793-4fba-9d5f-a631d3429dd9 None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:34.226111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f86e0e-4793-4fba-9d5f-a631d3429dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:34.227879 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f86e0e-4793-4fba-9d5f-a631d3429dd9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:34.230188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 80/161] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:33 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:34.235860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:34.296686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:34.399918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae52ccf-e076-488b-b39e-a634cc84a986 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:34.435721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4dd2-5179-4515-9f36-33f8511a78bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:34.436341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4dd2-5179-4515-9f36-33f8511a78bf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:34.436613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4dd2-5179-4515-9f36-33f8511a78bf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:34.436876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4dd2-5179-4515-9f36-33f8511a78bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:34.524325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da4dd2-5179-4515-9f36-33f8511a78bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:34.525778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da4dd2-5179-4515-9f36-33f8511a78bf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:34.566200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:34.657697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 82/162] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:34.663098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f86e0e-4793-4fba-9d5f-a631d3429dd9 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:34.744443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814d46f-0c1e-4564-8d58-350de89843fa None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:34.744766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814d46f-0c1e-4564-8d58-350de89843fa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:34.745041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814d46f-0c1e-4564-8d58-350de89843fa None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:34.746485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 83/163] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:34.752323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4602143-0ac3-4289-891c-3fd20d46dd46 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:34.752899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4602143-0ac3-4289-891c-3fd20d46dd46 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:34.752899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4602143-0ac3-4289-891c-3fd20d46dd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:34.784292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a4602143-0ac3-4289-891c-3fd20d46dd46 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:34.786059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a4602143-0ac3-4289-891c-3fd20d46dd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:34.823467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f86e0e-4793-4fba-9d5f-a631d3429dd9 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:34.917952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee19d16-8726-4917-9b96-f9e36a2c91a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:34.918597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee19d16-8726-4917-9b96-f9e36a2c91a9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:34.918873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee19d16-8726-4917-9b96-f9e36a2c91a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:34.919142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee19d16-8726-4917-9b96-f9e36a2c91a9 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:34.920331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee19d16-8726-4917-9b96-f9e36a2c91a9 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:34.921804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee19d16-8726-4917-9b96-f9e36a2c91a9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:34.923454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 81/164] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:34 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:34.923824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 84/165] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 13:24:34.924639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301e45-6475-414d-a9a5-1ea49bf36e09 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:34.924913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301e45-6475-414d-a9a5-1ea49bf36e09 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:34.925181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301e45-6475-414d-a9a5-1ea49bf36e09 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:34.929865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 82/166] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:34.938841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8e54-a873-4d3c-8b62-b9a117a4c665 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:34.939298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8e54-a873-4d3c-8b62-b9a117a4c665 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:34.939462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8e54-a873-4d3c-8b62-b9a117a4c665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:34.940443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74301e45-6475-414d-a9a5-1ea49bf36e09 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:34.975338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-32bc8e54-a873-4d3c-8b62-b9a117a4c665 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:34.977071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-32bc8e54-a873-4d3c-8b62-b9a117a4c665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:35.010765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:35.102202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:35.122190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 85/167] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:35.123346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc8e54-a873-4d3c-8b62-b9a117a4c665 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:35.213939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74301e45-6475-414d-a9a5-1ea49bf36e09 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:35.231956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc8e54-a873-4d3c-8b62-b9a117a4c665 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:35.280398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003100b9-2864-4fa0-9d50-20e7bc0bbdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:35.281322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003100b9-2864-4fa0-9d50-20e7bc0bbdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:35.281801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003100b9-2864-4fa0-9d50-20e7bc0bbdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:35.282292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003100b9-2864-4fa0-9d50-20e7bc0bbdd0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:35.294154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003100b9-2864-4fa0-9d50-20e7bc0bbdd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-003100b9-2864-4fa0-9d50-20e7bc0bbdd0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:35.299408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:35.300560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:35.301397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:35.303705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 86/168] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:24:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:35.305247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003100b9-2864-4fa0-9d50-20e7bc0bbdd0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:35.333974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:35.334593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:35.334869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:35.335172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:35.335899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] RBAC: Adding query filter params (role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=8d1b82d0c0604e3a98c466c903e60279, user.id=45a4a1663ad84ad1953e68ba332fad3f) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:35.336197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:35.337719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:35.355819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 83/169] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:34 2022] GET /identity/v3/role_assignments?user.id=45a4a1663ad84ad1953e68ba332fad3f&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 608 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:35.356597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:35.362912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:35.422708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003100b9-2864-4fa0-9d50-20e7bc0bbdd0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:35.462235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3579a2-ba03-41b0-a3a8-af8d6a7e0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:35.463284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3579a2-ba03-41b0-a3a8-af8d6a7e0129 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:35.463762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3579a2-ba03-41b0-a3a8-af8d6a7e0129 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:35.464234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3579a2-ba03-41b0-a3a8-af8d6a7e0129 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:35.476065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3579a2-ba03-41b0-a3a8-af8d6a7e0129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ad3579a2-ba03-41b0-a3a8-af8d6a7e0129 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:35.481554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:35.482780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:35.483557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:35.484640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 87/170] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:35.486918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a832-b808-4ae6-8586-69a09a4188c6 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:35.487364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a832-b808-4ae6-8586-69a09a4188c6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:35.487849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a832-b808-4ae6-8586-69a09a4188c6 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:35.489311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 88/171] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:35.498173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a832-b808-4ae6-8586-69a09a4188c6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:35.547705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:35.561439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26deff-520d-4d88-a0cc-fe4a90a8bd02 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:35.632331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bf75d-31e5-464b-8d30-559052822ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:35.634037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bf75d-31e5-464b-8d30-559052822ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:35.634318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bf75d-31e5-464b-8d30-559052822ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:35.634582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bf75d-31e5-464b-8d30-559052822ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:35.635465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94bf75d-31e5-464b-8d30-559052822ae5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:35.635900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94bf75d-31e5-464b-8d30-559052822ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:35.639088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94bf75d-31e5-464b-8d30-559052822ae5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:35.652795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 84/172] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:24:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:35.654483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad07e6-9d59-42a2-ab83-42fa2821167e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:35.654901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad07e6-9d59-42a2-ab83-42fa2821167e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:35.655366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad07e6-9d59-42a2-ab83-42fa2821167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:35.685022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-86ad07e6-9d59-42a2-ab83-42fa2821167e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:35.686757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-86ad07e6-9d59-42a2-ab83-42fa2821167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:35.687740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a832-b808-4ae6-8586-69a09a4188c6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:35.700604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:35.728716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:35.729749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:35.730379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:35.730848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:35.731720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:35.732236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:35.734116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:35.743473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 89/173] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:35.744609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:35.808503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 85/174] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:35.809708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad07e6-9d59-42a2-ab83-42fa2821167e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:35.929151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:35.960524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad07e6-9d59-42a2-ab83-42fa2821167e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:36.015237 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d71757-1c1a-433a-bfab-1eaacd445208 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:36.019242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b1521-1c9b-400d-8265-db5e4afe06ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:36.020438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1521-1c9b-400d-8265-db5e4afe06ad None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.022038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1521-1c9b-400d-8265-db5e4afe06ad None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.022651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1521-1c9b-400d-8265-db5e4afe06ad None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.029703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b1521-1c9b-400d-8265-db5e4afe06ad None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:36.031465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b1521-1c9b-400d-8265-db5e4afe06ad None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:36.033506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 86/175] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:35 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:36.036959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b1521-1c9b-400d-8265-db5e4afe06ad None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:36.088147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23003ad-f08d-41c7-b534-bcb0dfd0cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:36.089360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23003ad-f08d-41c7-b534-bcb0dfd0cd85 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.090907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23003ad-f08d-41c7-b534-bcb0dfd0cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.091392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23003ad-f08d-41c7-b534-bcb0dfd0cd85 None admin] PATH_INFO: `/v3/users/45a4a1663ad84ad1953e68ba332fad3f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.093477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23003ad-f08d-41c7-b534-bcb0dfd0cd85 None admin] RBAC: Authorizing `identity:get_user(user_id=45a4a1663ad84ad1953e68ba332fad3f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:36.095848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23003ad-f08d-41c7-b534-bcb0dfd0cd85 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:36.097851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 90/176] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:35 2022] GET /identity/v3/users/45a4a1663ad84ad1953e68ba332fad3f => generated 275 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:36.100224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23003ad-f08d-41c7-b534-bcb0dfd0cd85 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:36.186069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:36.221391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b1521-1c9b-400d-8265-db5e4afe06ad None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:36.264195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5dbe-a7c3-4c3a-b8ea-72057cd6e1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:36.265211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5dbe-a7c3-4c3a-b8ea-72057cd6e1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.265767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5dbe-a7c3-4c3a-b8ea-72057cd6e1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.266232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5dbe-a7c3-4c3a-b8ea-72057cd6e1f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.267945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23003ad-f08d-41c7-b534-bcb0dfd0cd85 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:36.282560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d5dbe-a7c3-4c3a-b8ea-72057cd6e1f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d52d5dbe-a7c3-4c3a-b8ea-72057cd6e1f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:36.287667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:36.288575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:36.289389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:36.291653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 87/177] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:36.297192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5dbe-a7c3-4c3a-b8ea-72057cd6e1f8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:36.306207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e6ead-1cd8-42dc-b097-b5b2baeb8d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:36.306829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e6ead-1cd8-42dc-b097-b5b2baeb8d76 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.307109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e6ead-1cd8-42dc-b097-b5b2baeb8d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.307383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e6ead-1cd8-42dc-b097-b5b2baeb8d76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.323134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:36.410535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e6ead-1cd8-42dc-b097-b5b2baeb8d76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:36.412771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5dbe-a7c3-4c3a-b8ea-72057cd6e1f8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:36.413808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e6ead-1cd8-42dc-b097-b5b2baeb8d76 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:36.448694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f059ac1-a5a9-42f3-9cb7-77926031f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:36.449220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f059ac1-a5a9-42f3-9cb7-77926031f0ab None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.449523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f059ac1-a5a9-42f3-9cb7-77926031f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.449802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f059ac1-a5a9-42f3-9cb7-77926031f0ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.492089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:36.540269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f059ac1-a5a9-42f3-9cb7-77926031f0ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:36.543009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f059ac1-a5a9-42f3-9cb7-77926031f0ab None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:36.557982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 91/178] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:36.559244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6246ae-b0e7-4f59-9ad1-441c56ea9944 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.559527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6246ae-b0e7-4f59-9ad1-441c56ea9944 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.559802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6246ae-b0e7-4f59-9ad1-441c56ea9944 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.561033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 92/179] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:36.565678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6246ae-b0e7-4f59-9ad1-441c56ea9944 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:36.571712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:36.683659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 88/180] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:36.685128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dcd03-842e-4d8f-8e0f-28966454403e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.685566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dcd03-842e-4d8f-8e0f-28966454403e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.685901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dcd03-842e-4d8f-8e0f-28966454403e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.687360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 89/181] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:36.692420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fde4f-0d07-4d53-a513-798d7ce706f1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.692700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fde4f-0d07-4d53-a513-798d7ce706f1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.692967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fde4f-0d07-4d53-a513-798d7ce706f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.696371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6246ae-b0e7-4f59-9ad1-441c56ea9944 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:36.713044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c60fde4f-0d07-4d53-a513-798d7ce706f1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:36.714170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c60fde4f-0d07-4d53-a513-798d7ce706f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:36.739280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:36.740004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.740280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.740531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.741203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:36.741593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:36.743005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:36.753044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 93/182] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:36.754592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:36.760451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:36.811521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 90/183] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:36.817450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fde4f-0d07-4d53-a513-798d7ce706f1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:36.868019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:36.883377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493de89-d021-4031-8446-ec3327b59a14 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:36.915958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925dc5e-0d3d-425f-a114-c1a8983727a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:36.916595 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925dc5e-0d3d-425f-a114-c1a8983727a8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.916873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925dc5e-0d3d-425f-a114-c1a8983727a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.917140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925dc5e-0d3d-425f-a114-c1a8983727a8 None admin] PATH_INFO: `/v3/projects/7a4d878448904154a14b5d3cfdd7e1b8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.918311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925dc5e-0d3d-425f-a114-c1a8983727a8 None admin] RBAC: Authorizing `identity:get_project(project_id=7a4d878448904154a14b5d3cfdd7e1b8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:36.919744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925dc5e-0d3d-425f-a114-c1a8983727a8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:36.926089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 94/184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:36 2022] GET /identity/v3/projects/7a4d878448904154a14b5d3cfdd7e1b8 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:36.926089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c4cef-80a6-47d3-904e-412b7c180331 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.926089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c4cef-80a6-47d3-904e-412b7c180331 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.926089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c4cef-80a6-47d3-904e-412b7c180331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.926417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fde4f-0d07-4d53-a513-798d7ce706f1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:36.942151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-196c4cef-80a6-47d3-904e-412b7c180331 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:36.943358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-196c4cef-80a6-47d3-904e-412b7c180331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:36.962036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e331d-f2ab-45e5-9593-fcecd5873668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:36.962668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e331d-f2ab-45e5-9593-fcecd5873668 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:36.962950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e331d-f2ab-45e5-9593-fcecd5873668 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:36.963277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e331d-f2ab-45e5-9593-fcecd5873668 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:36.964343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e331d-f2ab-45e5-9593-fcecd5873668 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:36.965839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e331d-f2ab-45e5-9593-fcecd5873668 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:36.967478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 91/185] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:36 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:36.968232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e331d-f2ab-45e5-9593-fcecd5873668 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.010525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:37.016159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 95/186] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:37.016995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c4cef-80a6-47d3-904e-412b7c180331 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.071957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e331d-f2ab-45e5-9593-fcecd5873668 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.081154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:37.106987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0dd70-6bb4-4c1a-a7ae-da45330d9171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.107855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0dd70-6bb4-4c1a-a7ae-da45330d9171 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.108197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0dd70-6bb4-4c1a-a7ae-da45330d9171 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.108386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0dd70-6bb4-4c1a-a7ae-da45330d9171 None admin] PATH_INFO: `/v3/users/ecff1398f8e14ef7b916c6c3c08928de` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.120684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c4cef-80a6-47d3-904e-412b7c180331 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.121648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0dd70-6bb4-4c1a-a7ae-da45330d9171 None admin] RBAC: Authorizing `identity:get_user(user_id=ecff1398f8e14ef7b916c6c3c08928de)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:37.123111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0dd70-6bb4-4c1a-a7ae-da45330d9171 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:37.124716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 92/187] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:36 2022] GET /identity/v3/users/ecff1398f8e14ef7b916c6c3c08928de => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:37.129321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295242-398c-424e-9e1d-5e0e06a10421 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.129321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295242-398c-424e-9e1d-5e0e06a10421 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.129321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295242-398c-424e-9e1d-5e0e06a10421 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.129321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 93/188] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:37.129321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5295242-398c-424e-9e1d-5e0e06a10421 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.158393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91617c54-84ae-476b-b506-6d6281e120e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.159019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617c54-84ae-476b-b506-6d6281e120e5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.159289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617c54-84ae-476b-b506-6d6281e120e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.159551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617c54-84ae-476b-b506-6d6281e120e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.160618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91617c54-84ae-476b-b506-6d6281e120e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:37.162072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91617c54-84ae-476b-b506-6d6281e120e5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:37.164118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 96/189] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:37.164941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91617c54-84ae-476b-b506-6d6281e120e5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.230125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:37.233383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5295242-398c-424e-9e1d-5e0e06a10421 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.266619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91617c54-84ae-476b-b506-6d6281e120e5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.266980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.267583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.267860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.268128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.268854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] RBAC: Adding query filter params (role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=7a4d878448904154a14b5d3cfdd7e1b8, user.id=45a4a1663ad84ad1953e68ba332fad3f) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:37.269151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:37.270613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:37.277133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 94/190] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:37 2022] GET /identity/v3/role_assignments?user.id=45a4a1663ad84ad1953e68ba332fad3f&scope.project.id=7a4d878448904154a14b5d3cfdd7e1b8&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:37.277428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.310644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.311274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.311550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.311818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.312573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] RBAC: Adding query filter params (role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=8d1b82d0c0604e3a98c466c903e60279, user.id=45a4a1663ad84ad1953e68ba332fad3f) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:37.312864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:37.314479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:37.315079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:37.321177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 97/191] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:37 2022] GET /identity/v3/role_assignments?user.id=45a4a1663ad84ad1953e68ba332fad3f&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:37.321943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.420929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6804a-f570-46af-beea-f7083d387831 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.459991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4c1c8-72ef-41b1-8bd7-27c8e656a251 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.464892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e07f9a-5906-40bf-9d64-7f2556b06eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.466044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e07f9a-5906-40bf-9d64-7f2556b06eca None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.466701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e07f9a-5906-40bf-9d64-7f2556b06eca None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.467457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e07f9a-5906-40bf-9d64-7f2556b06eca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.486008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e07f9a-5906-40bf-9d64-7f2556b06eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-80e07f9a-5906-40bf-9d64-7f2556b06eca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:37.492406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:37.494090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 95/192] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:24:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:37.495398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131584f-d7f6-4d9b-a6c4-be131a4c5904 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.497602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131584f-d7f6-4d9b-a6c4-be131a4c5904 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.498078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131584f-d7f6-4d9b-a6c4-be131a4c5904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.515926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:37.522170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fbe65-4292-4cd1-883a-e06e7285f77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.523787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fbe65-4292-4cd1-883a-e06e7285f77e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.524067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fbe65-4292-4cd1-883a-e06e7285f77e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.524345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fbe65-4292-4cd1-883a-e06e7285f77e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.524906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2131584f-d7f6-4d9b-a6c4-be131a4c5904 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:37.526147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2131584f-d7f6-4d9b-a6c4-be131a4c5904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:37.619398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fbe65-4292-4cd1-883a-e06e7285f77e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:37.620862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fbe65-4292-4cd1-883a-e06e7285f77e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:37.624503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 96/193] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:37.625345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131584f-d7f6-4d9b-a6c4-be131a4c5904 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.655645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:37.728200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 98/194] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:37.734274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fbe65-4292-4cd1-883a-e06e7285f77e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.745616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131584f-d7f6-4d9b-a6c4-be131a4c5904 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.773079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:37.795966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec671f-1273-4e63-ae3b-548194a90c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.796568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec671f-1273-4e63-ae3b-548194a90c67 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.796836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec671f-1273-4e63-ae3b-548194a90c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.797098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec671f-1273-4e63-ae3b-548194a90c67 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.797755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec671f-1273-4e63-ae3b-548194a90c67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:37.799197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec671f-1273-4e63-ae3b-548194a90c67 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:37.822277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 97/195] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:24:37 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:37.823185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec671f-1273-4e63-ae3b-548194a90c67 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.862537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fbe65-4292-4cd1-883a-e06e7285f77e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.903480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.904392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.904667 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.904931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:37.905817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:37.906992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:37.908684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:37.915478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:37.923814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 99/196] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:24:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:37.923814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:37.951262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec671f-1273-4e63-ae3b-548194a90c67 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:37.992715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379b4cb-96e7-4008-b80e-9ab1ce721e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:37.993055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379b4cb-96e7-4008-b80e-9ab1ce721e6c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:37.994663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379b4cb-96e7-4008-b80e-9ab1ce721e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:37.994663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379b4cb-96e7-4008-b80e-9ab1ce721e6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.009344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379b4cb-96e7-4008-b80e-9ab1ce721e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4379b4cb-96e7-4008-b80e-9ab1ce721e6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:38.014608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:38.015466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:38.016298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 98/197] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:38.016964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379b4cb-96e7-4008-b80e-9ab1ce721e6c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:38.033941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:38.042590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42432d-d251-4c53-9550-9efde79473ec None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:38.090918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359543a-cc51-4f10-b252-44b34a158a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:38.091573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359543a-cc51-4f10-b252-44b34a158a87 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:38.091847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359543a-cc51-4f10-b252-44b34a158a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:38.092115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359543a-cc51-4f10-b252-44b34a158a87 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.094293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359543a-cc51-4f10-b252-44b34a158a87 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:38.096359 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359543a-cc51-4f10-b252-44b34a158a87 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:38.098043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 100/198] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:37 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:38.098255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359543a-cc51-4f10-b252-44b34a158a87 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:38.150189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379b4cb-96e7-4008-b80e-9ab1ce721e6c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:38.198304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66442892-ae25-4d47-b2a7-403abddfe268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:38.198947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66442892-ae25-4d47-b2a7-403abddfe268 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:38.199220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66442892-ae25-4d47-b2a7-403abddfe268 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:38.199489 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66442892-ae25-4d47-b2a7-403abddfe268 None admin] PATH_INFO: `/v3/users/ecff1398f8e14ef7b916c6c3c08928de` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.200637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66442892-ae25-4d47-b2a7-403abddfe268 None admin] RBAC: Authorizing `identity:get_user(user_id=ecff1398f8e14ef7b916c6c3c08928de)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:38.202285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66442892-ae25-4d47-b2a7-403abddfe268 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:38.203900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 99/199] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:38 2022] GET /identity/v3/users/ecff1398f8e14ef7b916c6c3c08928de => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:38.204801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66442892-ae25-4d47-b2a7-403abddfe268 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:38.209559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:38.240391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359543a-cc51-4f10-b252-44b34a158a87 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:38.283905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a0689-70d4-496e-81eb-845433606208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:38.284467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0689-70d4-496e-81eb-845433606208 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:38.284750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0689-70d4-496e-81eb-845433606208 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:38.285018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a0689-70d4-496e-81eb-845433606208 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.285716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a0689-70d4-496e-81eb-845433606208 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:38.286017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a0689-70d4-496e-81eb-845433606208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:38.287460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a0689-70d4-496e-81eb-845433606208 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:38.298797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 101/200] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:38.299417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a0689-70d4-496e-81eb-845433606208 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:38.328531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:38.347315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66442892-ae25-4d47-b2a7-403abddfe268 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:38.394296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:38.395306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:38.395801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:38.396281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.397347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec, user.id=ecff1398f8e14ef7b916c6c3c08928de) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:38.397905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:38.399649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:38.406596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 100/201] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:38 2022] GET /identity/v3/role_assignments?user.id=ecff1398f8e14ef7b916c6c3c08928de&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:38.407983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:38.415019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a0689-70d4-496e-81eb-845433606208 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:38.457783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6294-e5c6-4ea9-8c6b-9b1790db4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:38.458657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6294-e5c6-4ea9-8c6b-9b1790db4c45 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:38.458938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6294-e5c6-4ea9-8c6b-9b1790db4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:38.459263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6294-e5c6-4ea9-8c6b-9b1790db4c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.492975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:38.534991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c0f15-cd3b-4ae7-bb38-42f6bc696cd9 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:38.538698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d6294-e5c6-4ea9-8c6b-9b1790db4c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:38.540270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d6294-e5c6-4ea9-8c6b-9b1790db4c45 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:38.567995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:38.579026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf37be-dd47-4687-85d6-ffc81ade4f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:38.581387 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf37be-dd47-4687-85d6-ffc81ade4f26 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:38.581938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf37be-dd47-4687-85d6-ffc81ade4f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:38.582434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf37be-dd47-4687-85d6-ffc81ade4f26 None admin] PATH_INFO: `/v3/users/e24fdc860b5246a6a2242933f5e96c75` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.583969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf37be-dd47-4687-85d6-ffc81ade4f26 None admin] RBAC: Authorizing `identity:get_user(user_id=e24fdc860b5246a6a2242933f5e96c75)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:38.586255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf37be-dd47-4687-85d6-ffc81ade4f26 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:38.588210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 101/202] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:38 2022] GET /identity/v3/users/e24fdc860b5246a6a2242933f5e96c75 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:38.605772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf37be-dd47-4687-85d6-ffc81ade4f26 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:38.697281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 102/203] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:38.711025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6294-e5c6-4ea9-8c6b-9b1790db4c45 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:38.745972 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf37be-dd47-4687-85d6-ffc81ade4f26 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:38.789294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dbdf0-c6c7-4b41-847b-5360e97ae934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:38.789828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dbdf0-c6c7-4b41-847b-5360e97ae934 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:38.790101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dbdf0-c6c7-4b41-847b-5360e97ae934 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:38.790366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dbdf0-c6c7-4b41-847b-5360e97ae934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.796652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:38.819127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6294-e5c6-4ea9-8c6b-9b1790db4c45 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:38.850521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:38.869457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c274e-48ea-4490-bb58-8032ae574082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:38.870473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c274e-48ea-4490-bb58-8032ae574082 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:38.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c274e-48ea-4490-bb58-8032ae574082 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:38.871476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c274e-48ea-4490-bb58-8032ae574082 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:38.872844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c274e-48ea-4490-bb58-8032ae574082 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:38.874593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c274e-48ea-4490-bb58-8032ae574082 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:38.876403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 103/204] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:38 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:38.893000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c274e-48ea-4490-bb58-8032ae574082 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:38.941820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dbdf0-c6c7-4b41-847b-5360e97ae934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:38.943311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dbdf0-c6c7-4b41-847b-5360e97ae934 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:39.084227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c274e-48ea-4490-bb58-8032ae574082 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:39.087110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 102/205] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:39.094504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dbdf0-c6c7-4b41-847b-5360e97ae934 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:39.120830 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:39.121482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:39.121755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:39.122017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:39.122805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] RBAC: Adding query filter params (role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec, user.id=ecff1398f8e14ef7b916c6c3c08928de) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:39.123252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:39.124699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:39.130760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 104/206] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:38 2022] GET /identity/v3/role_assignments?user.id=ecff1398f8e14ef7b916c6c3c08928de&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:39.153337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:39.226688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dbdf0-c6c7-4b41-847b-5360e97ae934 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:39.268379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6687e2-e13d-4ed2-8dc1-7370449cfa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:39.269040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6687e2-e13d-4ed2-8dc1-7370449cfa13 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:39.269393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6687e2-e13d-4ed2-8dc1-7370449cfa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:39.269674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6687e2-e13d-4ed2-8dc1-7370449cfa13 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:39.271010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6687e2-e13d-4ed2-8dc1-7370449cfa13 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:39.272639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6687e2-e13d-4ed2-8dc1-7370449cfa13 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:39.274578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 103/207] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:39 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:39.283765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:39.365534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:39.416093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b64311-335b-42bd-9174-429436c8afde None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:39.456823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f07b0-d059-41e4-82d0-765256a3a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:39.457870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f07b0-d059-41e4-82d0-765256a3a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:39.458355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f07b0-d059-41e4-82d0-765256a3a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:39.458826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f07b0-d059-41e4-82d0-765256a3a5bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:39.459786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f07b0-d059-41e4-82d0-765256a3a5bf None admin] RBAC: Adding query filter params (role.id=9ee2cd43468941e28f08b0be54868574, scope.project.id=8d1b82d0c0604e3a98c466c903e60279, user.id=e24fdc860b5246a6a2242933f5e96c75) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:39.460358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f07b0-d059-41e4-82d0-765256a3a5bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:39.462020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f07b0-d059-41e4-82d0-765256a3a5bf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:39.468513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 105/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:39 2022] GET /identity/v3/role_assignments?user.id=e24fdc860b5246a6a2242933f5e96c75&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=9ee2cd43468941e28f08b0be54868574 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:40.232080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e04c1e-1e93-498d-91bb-0c744b96d4ff None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.232870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e04c1e-1e93-498d-91bb-0c744b96d4ff None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.233361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e04c1e-1e93-498d-91bb-0c744b96d4ff None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.235996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 104/209] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:40.241944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef20fe-9c8c-4c83-89c1-e35c5fec7541 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.242392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef20fe-9c8c-4c83-89c1-e35c5fec7541 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.242485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef20fe-9c8c-4c83-89c1-e35c5fec7541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.264566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c7ef20fe-9c8c-4c83-89c1-e35c5fec7541 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:40.265852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c7ef20fe-9c8c-4c83-89c1-e35c5fec7541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:40.336855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 106/210] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:40.360113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e04c1e-1e93-498d-91bb-0c744b96d4ff None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:40.474243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e04c1e-1e93-498d-91bb-0c744b96d4ff None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:40.477003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:40.505478 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07035e-1627-458c-8bfc-78c574eb72d8 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.505758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07035e-1627-458c-8bfc-78c574eb72d8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.506082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07035e-1627-458c-8bfc-78c574eb72d8 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.508261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 107/211] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:40.514559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d97844-5922-4337-8131-c41e7aefa9ce None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.514956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d97844-5922-4337-8131-c41e7aefa9ce None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.515323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d97844-5922-4337-8131-c41e7aefa9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.517864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a98bd-ddbf-4288-8ab3-c74d95f64088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:40.518688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a98bd-ddbf-4288-8ab3-c74d95f64088 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.519096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a98bd-ddbf-4288-8ab3-c74d95f64088 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.519504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a98bd-ddbf-4288-8ab3-c74d95f64088 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.531269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a98bd-ddbf-4288-8ab3-c74d95f64088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-da9a98bd-ddbf-4288-8ab3-c74d95f64088 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:40.536527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:40.537652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:40.538692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:40.539869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c4d97844-5922-4337-8131-c41e7aefa9ce None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:40.541076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 105/212] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:40.541335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c4d97844-5922-4337-8131-c41e7aefa9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:40.548149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a98bd-ddbf-4288-8ab3-c74d95f64088 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:40.598428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:40.631491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 108/213] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:40.653103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d97844-5922-4337-8131-c41e7aefa9ce None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:40.686807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a98bd-ddbf-4288-8ab3-c74d95f64088 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:40.731939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:40.733593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.734022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.734439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.735627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:40.736093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:40.738312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:40.750066 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 106/214] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:40.756203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:40.767027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d97844-5922-4337-8131-c41e7aefa9ce None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:40.789589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:40.802636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d91217-bc04-459e-9c1f-7d3a8878c1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:40.803385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d91217-bc04-459e-9c1f-7d3a8878c1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.804193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d91217-bc04-459e-9c1f-7d3a8878c1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.804461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d91217-bc04-459e-9c1f-7d3a8878c1c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.805595 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d91217-bc04-459e-9c1f-7d3a8878c1c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:40.807032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d91217-bc04-459e-9c1f-7d3a8878c1c8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:40.808594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 109/215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:40.811349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ed713-bee1-4ab8-801e-2df0f0f9a9cf None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.811703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ed713-bee1-4ab8-801e-2df0f0f9a9cf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.812082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ed713-bee1-4ab8-801e-2df0f0f9a9cf None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.814195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 110/216] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:40.814902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ed713-bee1-4ab8-801e-2df0f0f9a9cf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:40.873819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:40.878623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff087fc-c0e5-40b8-8cc9-182d23b7661c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:40.915593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e96a91-43da-4060-a996-95178ba4d779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:40.916240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e96a91-43da-4060-a996-95178ba4d779 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.916521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e96a91-43da-4060-a996-95178ba4d779 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.916793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e96a91-43da-4060-a996-95178ba4d779 None admin] PATH_INFO: `/v3/users/bb1185ec899e4715af180f73c65d68b8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.931533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ed713-bee1-4ab8-801e-2df0f0f9a9cf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:40.933095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e96a91-43da-4060-a996-95178ba4d779 None admin] RBAC: Authorizing `identity:get_user(user_id=bb1185ec899e4715af180f73c65d68b8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:40.934644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e96a91-43da-4060-a996-95178ba4d779 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:40.940801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 107/217] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:40 2022] GET /identity/v3/users/bb1185ec899e4715af180f73c65d68b8 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:40.940801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b846724-0c16-4476-ba5c-a3863c6cd91c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.940801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b846724-0c16-4476-ba5c-a3863c6cd91c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.940801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b846724-0c16-4476-ba5c-a3863c6cd91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.956796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5b846724-0c16-4476-ba5c-a3863c6cd91c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:40.958024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5b846724-0c16-4476-ba5c-a3863c6cd91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:40.970032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf3fdf-a3d9-46bd-8f4d-484f162abc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:40.970655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf3fdf-a3d9-46bd-8f4d-484f162abc78 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:40.970929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf3fdf-a3d9-46bd-8f4d-484f162abc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:40.971193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf3fdf-a3d9-46bd-8f4d-484f162abc78 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:40.971844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf3fdf-a3d9-46bd-8f4d-484f162abc78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:40.973305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf3fdf-a3d9-46bd-8f4d-484f162abc78 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:40.998902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 111/218] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:24:40 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:41.002179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf3fdf-a3d9-46bd-8f4d-484f162abc78 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:41.065321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 108/219] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:41.082017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b846724-0c16-4476-ba5c-a3863c6cd91c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:41.099486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:41.144046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf3fdf-a3d9-46bd-8f4d-484f162abc78 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:41.147234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:41.192603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa95273-a549-4a1d-9b35-7777744716c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:41.193559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95273-a549-4a1d-9b35-7777744716c4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:41.193834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95273-a549-4a1d-9b35-7777744716c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:41.194102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95273-a549-4a1d-9b35-7777744716c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:41.201511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b846724-0c16-4476-ba5c-a3863c6cd91c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:41.239407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322e89f-f3c1-48de-9aab-eb0537b8b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:41.240063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322e89f-f3c1-48de-9aab-eb0537b8b039 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:41.240820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322e89f-f3c1-48de-9aab-eb0537b8b039 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:41.240820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322e89f-f3c1-48de-9aab-eb0537b8b039 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:41.254605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322e89f-f3c1-48de-9aab-eb0537b8b039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-7322e89f-f3c1-48de-9aab-eb0537b8b039 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:41.259505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:41.260394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:41.261170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:41.264292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 109/220] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:41.267712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322e89f-f3c1-48de-9aab-eb0537b8b039 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:41.290636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa95273-a549-4a1d-9b35-7777744716c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:41.291299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa95273-a549-4a1d-9b35-7777744716c4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:41.372085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:41.408440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322e89f-f3c1-48de-9aab-eb0537b8b039 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:41.425269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 112/221] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:41.432016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa95273-a549-4a1d-9b35-7777744716c4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:41.436512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:41.454985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:41.455993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:41.456465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:41.456947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:41.458392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:41.458898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:41.461092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:41.476268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 110/222] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:41.480393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:41.555870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa95273-a549-4a1d-9b35-7777744716c4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:41.600422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef55c8-9d35-4153-8580-adc17f9fbe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:41.601804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef55c8-9d35-4153-8580-adc17f9fbe57 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:41.602302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef55c8-9d35-4153-8580-adc17f9fbe57 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:41.603035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef55c8-9d35-4153-8580-adc17f9fbe57 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:41.605185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef55c8-9d35-4153-8580-adc17f9fbe57 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:41.607781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef55c8-9d35-4153-8580-adc17f9fbe57 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:41.609709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 113/223] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:41 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:41.611299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9e26-4700-4b88-981d-878ffa4f4856 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:41.611786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9e26-4700-4b88-981d-878ffa4f4856 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:41.612247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b9e26-4700-4b88-981d-878ffa4f4856 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:41.613715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 114/224] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:41.618432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b9e26-4700-4b88-981d-878ffa4f4856 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:41.632730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4dcc17-f710-497e-b587-7a5ee3173bd8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:41.671470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b3b36-dc3f-4728-8d04-48c2d10a626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:41.672086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3b36-dc3f-4728-8d04-48c2d10a626b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:41.672364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3b36-dc3f-4728-8d04-48c2d10a626b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:41.672631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3b36-dc3f-4728-8d04-48c2d10a626b None admin] PATH_INFO: `/v3/users/ecff1398f8e14ef7b916c6c3c08928de` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:41.674612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b3b36-dc3f-4728-8d04-48c2d10a626b None admin] RBAC: Authorizing `identity:get_user(user_id=ecff1398f8e14ef7b916c6c3c08928de)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:41.676111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b3b36-dc3f-4728-8d04-48c2d10a626b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:41.678014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 111/225] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:41 2022] GET /identity/v3/users/ecff1398f8e14ef7b916c6c3c08928de => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:41.679164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274afbe-77ba-40e2-a557-383e8462b506 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:41.679442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274afbe-77ba-40e2-a557-383e8462b506 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:41.679710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274afbe-77ba-40e2-a557-383e8462b506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:41.687661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:41.699982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d274afbe-77ba-40e2-a557-383e8462b506 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:41.701157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d274afbe-77ba-40e2-a557-383e8462b506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:41.710689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:41.748080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b9e26-4700-4b88-981d-878ffa4f4856 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:41.789350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 112/226] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:41.789350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274afbe-77ba-40e2-a557-383e8462b506 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:41.805711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:41.806653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:41.807096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:41.807565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:41.808435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] RBAC: Adding query filter params (role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec, user.id=bb1185ec899e4715af180f73c65d68b8) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:41.808984 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:41.813381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:41.825833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 115/227] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:41 2022] GET /identity/v3/role_assignments?user.id=bb1185ec899e4715af180f73c65d68b8&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:41.825833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:41.997576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274afbe-77ba-40e2-a557-383e8462b506 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:42.017617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721db52e-2dbb-44ea-abfe-3090cf14d6ca None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:42.039942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a934b3-ec12-46b4-917b-d863df330680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:42.040581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a934b3-ec12-46b4-917b-d863df330680 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:42.040855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a934b3-ec12-46b4-917b-d863df330680 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:42.041121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a934b3-ec12-46b4-917b-d863df330680 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:42.043073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a934b3-ec12-46b4-917b-d863df330680 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:42.044521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a934b3-ec12-46b4-917b-d863df330680 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:42.046168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 113/228] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:41 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:42.051925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a934b3-ec12-46b4-917b-d863df330680 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:42.055912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:42.099809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e75c-9404-4ddc-9c7e-775be67e5296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:42.100536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e75c-9404-4ddc-9c7e-775be67e5296 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:42.101355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e75c-9404-4ddc-9c7e-775be67e5296 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:42.101621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e75c-9404-4ddc-9c7e-775be67e5296 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:42.109112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:42.131002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17e75c-9404-4ddc-9c7e-775be67e5296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-4b17e75c-9404-4ddc-9c7e-775be67e5296 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:42.136370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:42.137487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:42.138497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:42.145960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 116/229] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:24:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:42.152579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e75c-9404-4ddc-9c7e-775be67e5296 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:42.162974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a934b3-ec12-46b4-917b-d863df330680 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:42.214612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df6384-cce3-4276-960c-42262d0b8d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:42.215276 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df6384-cce3-4276-960c-42262d0b8d55 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:42.215545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df6384-cce3-4276-960c-42262d0b8d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:42.215807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df6384-cce3-4276-960c-42262d0b8d55 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/users/ecff1398f8e14ef7b916c6c3c08928de/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:42.220519 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df6384-cce3-4276-960c-42262d0b8d55 None admin] RBAC: Authorizing `identity:create_grant(project_id=28871a3f7ee34b03bf66682bb0a217ec, user_id=ecff1398f8e14ef7b916c6c3c08928de, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:42.226992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df6384-cce3-4276-960c-42262d0b8d55 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:42.249348 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 114/230] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:42 2022] PUT /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/users/ecff1398f8e14ef7b916c6c3c08928de/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:24:42.307129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e75c-9404-4ddc-9c7e-775be67e5296 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:42.352566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:42.353182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:42.353489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:42.353876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:42.354597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:42.354933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:42.356390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:42.367299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 117/231] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:24:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:42.373070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df6384-cce3-4276-960c-42262d0b8d55 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:42.467824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:42.515203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df6384-cce3-4276-960c-42262d0b8d55 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:42.552465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428a82-4994-412f-9db1-9eefd8e9bd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:42.553099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428a82-4994-412f-9db1-9eefd8e9bd91 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:42.553413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428a82-4994-412f-9db1-9eefd8e9bd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:42.553712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428a82-4994-412f-9db1-9eefd8e9bd91 None admin] PATH_INFO: `/v3/users/ecff1398f8e14ef7b916c6c3c08928de` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:42.554924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19428a82-4994-412f-9db1-9eefd8e9bd91 None admin] RBAC: Authorizing `identity:get_user(user_id=ecff1398f8e14ef7b916c6c3c08928de)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:42.556483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19428a82-4994-412f-9db1-9eefd8e9bd91 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:42.558423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 115/232] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:42 2022] GET /identity/v3/users/ecff1398f8e14ef7b916c6c3c08928de => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:42.563570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:42.634455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:42.683921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039ca1e-771c-48cb-b83e-cca8d53a9cff None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:42.731564 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcb24b-4cb1-4e4c-877a-2eb3049bc1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:42.732219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcb24b-4cb1-4e4c-877a-2eb3049bc1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:42.732927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcb24b-4cb1-4e4c-877a-2eb3049bc1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:42.733208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcb24b-4cb1-4e4c-877a-2eb3049bc1d1 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:42.734358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcb24b-4cb1-4e4c-877a-2eb3049bc1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:42.735812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcb24b-4cb1-4e4c-877a-2eb3049bc1d1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:42.737783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 118/233] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:42 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:42.743632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428a82-4994-412f-9db1-9eefd8e9bd91 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:42.884255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428a82-4994-412f-9db1-9eefd8e9bd91 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:42.912828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:42.918507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff3473-ae80-4b05-aefc-85959df17590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:42.919171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3473-ae80-4b05-aefc-85959df17590 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:42.919449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3473-ae80-4b05-aefc-85959df17590 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:42.919723 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3473-ae80-4b05-aefc-85959df17590 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/users/ecff1398f8e14ef7b916c6c3c08928de/roles/d0ccb065ab134c66b5033b67d0bb86b6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:42.921916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff3473-ae80-4b05-aefc-85959df17590 None admin] RBAC: Authorizing `identity:create_grant(project_id=28871a3f7ee34b03bf66682bb0a217ec, user_id=ecff1398f8e14ef7b916c6c3c08928de, role_id=d0ccb065ab134c66b5033b67d0bb86b6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:42.923635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff3473-ae80-4b05-aefc-85959df17590 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:42.940609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 116/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:42 2022] PUT /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/users/ecff1398f8e14ef7b916c6c3c08928de/roles/d0ccb065ab134c66b5033b67d0bb86b6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:43.550659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895668d3-dbdc-4e86-accb-1828327c061d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:43.551154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895668d3-dbdc-4e86-accb-1828327c061d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:43.551154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895668d3-dbdc-4e86-accb-1828327c061d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:43.552714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 119/235] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:43.558310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b02d6-14c2-4d0f-8150-628d63888dfa None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:43.558737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b02d6-14c2-4d0f-8150-628d63888dfa None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:43.558902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b02d6-14c2-4d0f-8150-628d63888dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:43.577490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-528b02d6-14c2-4d0f-8150-628d63888dfa None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:43.578648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-528b02d6-14c2-4d0f-8150-628d63888dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:43.646270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 117/236] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:43.668561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895668d3-dbdc-4e86-accb-1828327c061d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:43.800234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a1f02-83c7-473c-b630-8cb8753051ff None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:43.800559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a1f02-83c7-473c-b630-8cb8753051ff None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:43.800825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a1f02-83c7-473c-b630-8cb8753051ff None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:43.802166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 118/237] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:43.808469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4f5f1-4225-4dfb-b152-633f1dd9df95 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:43.809784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4f5f1-4225-4dfb-b152-633f1dd9df95 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:43.809784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4f5f1-4225-4dfb-b152-633f1dd9df95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:43.832157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895668d3-dbdc-4e86-accb-1828327c061d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:43.841036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7ce4f5f1-4225-4dfb-b152-633f1dd9df95 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:43.842588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7ce4f5f1-4225-4dfb-b152-633f1dd9df95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:43.874600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:43.895593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2e45-7e81-4345-9349-1d13416093ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:43.896259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2e45-7e81-4345-9349-1d13416093ec None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:43.896533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2e45-7e81-4345-9349-1d13416093ec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:43.896796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2e45-7e81-4345-9349-1d13416093ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:43.922682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c2e45-7e81-4345-9349-1d13416093ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:43.931018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 119/238] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-686c2e45-7e81-4345-9349-1d13416093ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:43.932198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:43.933061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:43.933867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:43.934928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 120/239] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:43.944950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4f5f1-4225-4dfb-b152-633f1dd9df95 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:43.963153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2e45-7e81-4345-9349-1d13416093ec None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:44.033620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:44.131484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4f5f1-4225-4dfb-b152-633f1dd9df95 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:44.137752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2e45-7e81-4345-9349-1d13416093ec None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:44.171483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:44.172451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:44.172927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:44.173574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:44.174438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:44.174957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:44.176551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:44.186963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 120/240] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:44.196607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:44.214992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d099a1-c768-42c7-bcf8-935b731e3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:44.215683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d099a1-c768-42c7-bcf8-935b731e3def None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:44.215958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d099a1-c768-42c7-bcf8-935b731e3def None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:44.216220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d099a1-c768-42c7-bcf8-935b731e3def None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:44.221883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d099a1-c768-42c7-bcf8-935b731e3def None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:44.223397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d099a1-c768-42c7-bcf8-935b731e3def None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:44.225151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 121/241] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:44.230886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d099a1-c768-42c7-bcf8-935b731e3def None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:44.314710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0ea11-6bd4-463f-a821-9b362361ab4c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:44.316142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:44.329817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:44.351648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233d9a7-1d5c-47f1-9040-4cbf64ae4bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:44.352609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d9a7-1d5c-47f1-9040-4cbf64ae4bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:44.353193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d9a7-1d5c-47f1-9040-4cbf64ae4bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:44.353736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d9a7-1d5c-47f1-9040-4cbf64ae4bc4 None admin] PATH_INFO: `/v3/users/61593889fa6142e28a1a49b93bc09908` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:44.369313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233d9a7-1d5c-47f1-9040-4cbf64ae4bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=61593889fa6142e28a1a49b93bc09908)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:44.371070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233d9a7-1d5c-47f1-9040-4cbf64ae4bc4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:44.372756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 121/242] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:44 2022] GET /identity/v3/users/61593889fa6142e28a1a49b93bc09908 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:44.373958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab35bd-ba8e-4ba2-9304-842b5ede36b5 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:44.374281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab35bd-ba8e-4ba2-9304-842b5ede36b5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:44.374608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab35bd-ba8e-4ba2-9304-842b5ede36b5 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:44.374729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d099a1-c768-42c7-bcf8-935b731e3def None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:44.375827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 122/243] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:44.380607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab35bd-ba8e-4ba2-9304-842b5ede36b5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:44.419627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe18005-94f2-462a-986e-de57a475f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:44.420161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe18005-94f2-462a-986e-de57a475f043 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:44.420435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe18005-94f2-462a-986e-de57a475f043 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:44.420697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe18005-94f2-462a-986e-de57a475f043 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:44.421371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe18005-94f2-462a-986e-de57a475f043 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:44.422821 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe18005-94f2-462a-986e-de57a475f043 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:44.453676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 122/244] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:24:44 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:24:44.455081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183151b4-fc87-4596-ba19-f88f170f5876 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:44.455476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183151b4-fc87-4596-ba19-f88f170f5876 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:44.455756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183151b4-fc87-4596-ba19-f88f170f5876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:44.499408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-183151b4-fc87-4596-ba19-f88f170f5876 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:44.501329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-183151b4-fc87-4596-ba19-f88f170f5876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:44.537694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab35bd-ba8e-4ba2-9304-842b5ede36b5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:44.605069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304630ad-22e6-4813-a11d-e4979d9fa834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:44.605738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304630ad-22e6-4813-a11d-e4979d9fa834 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:44.605976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304630ad-22e6-4813-a11d-e4979d9fa834 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:44.606249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304630ad-22e6-4813-a11d-e4979d9fa834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:44.655599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 123/245] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:44.677032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:44.680648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183151b4-fc87-4596-ba19-f88f170f5876 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:44.717020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304630ad-22e6-4813-a11d-e4979d9fa834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:44.718911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304630ad-22e6-4813-a11d-e4979d9fa834 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:44.735251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:44.810162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183151b4-fc87-4596-ba19-f88f170f5876 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:44.851422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 123/246] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:44.856511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304630ad-22e6-4813-a11d-e4979d9fa834 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:44.859547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca15f44-14b2-48f0-974b-4f54aa485681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:44.860504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca15f44-14b2-48f0-974b-4f54aa485681 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:44.861016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca15f44-14b2-48f0-974b-4f54aa485681 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:44.861525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca15f44-14b2-48f0-974b-4f54aa485681 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:44.875022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca15f44-14b2-48f0-974b-4f54aa485681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3ca15f44-14b2-48f0-974b-4f54aa485681 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:44.882791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:44.883714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:44.884483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:44.887372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 124/247] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:44.893395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca15f44-14b2-48f0-974b-4f54aa485681 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:44.939360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:44.977969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304630ad-22e6-4813-a11d-e4979d9fa834 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:45.019702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27802a76-67ef-4f1d-a4e8-a11bed25e576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:45.020368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:45.021929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27802a76-67ef-4f1d-a4e8-a11bed25e576 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.023178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27802a76-67ef-4f1d-a4e8-a11bed25e576 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.025586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca15f44-14b2-48f0-974b-4f54aa485681 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:45.025884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27802a76-67ef-4f1d-a4e8-a11bed25e576 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.028160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27802a76-67ef-4f1d-a4e8-a11bed25e576 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:45.029962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27802a76-67ef-4f1d-a4e8-a11bed25e576 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:45.032363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 124/248] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:44 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:45.035152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f665d7-43b4-4c9b-abbe-17e66774980d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.035678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f665d7-43b4-4c9b-abbe-17e66774980d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.036217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f665d7-43b4-4c9b-abbe-17e66774980d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.038159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 125/249] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:45.039193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f665d7-43b4-4c9b-abbe-17e66774980d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:45.064515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b239253-4c49-4541-95fb-fa5441f24b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:45.065132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b239253-4c49-4541-95fb-fa5441f24b0f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.065435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b239253-4c49-4541-95fb-fa5441f24b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.065705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b239253-4c49-4541-95fb-fa5441f24b0f None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.066368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b239253-4c49-4541-95fb-fa5441f24b0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:45.066665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b239253-4c49-4541-95fb-fa5441f24b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:45.068063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b239253-4c49-4541-95fb-fa5441f24b0f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:45.077999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 125/250] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:45.079146 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863c62-e2fd-4627-8fda-7f9b95ce5baf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.079430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863c62-e2fd-4627-8fda-7f9b95ce5baf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.079809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863c62-e2fd-4627-8fda-7f9b95ce5baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.100217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-48863c62-e2fd-4627-8fda-7f9b95ce5baf None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:45.102003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-48863c62-e2fd-4627-8fda-7f9b95ce5baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:45.170663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f665d7-43b4-4c9b-abbe-17e66774980d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:45.189013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 126/251] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:45.194116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48863c62-e2fd-4627-8fda-7f9b95ce5baf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:45.229305 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:45.229982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.230265 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.230538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.231338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] RBAC: Adding query filter params (role.id=9ee2cd43468941e28f08b0be54868574, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec, user.id=61593889fa6142e28a1a49b93bc09908) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:45.232220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:45.234355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:45.243082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 126/252] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:45 2022] GET /identity/v3/role_assignments?user.id=61593889fa6142e28a1a49b93bc09908&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=9ee2cd43468941e28f08b0be54868574 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:45.243764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:45.247521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:45.343284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:45.383558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48863c62-e2fd-4627-8fda-7f9b95ce5baf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:45.385572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602307-1959-45fb-a0ad-2167fe4db32c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:45.428639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d6bec-684d-424d-8c81-54bc82f31096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:45.429666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d6bec-684d-424d-8c81-54bc82f31096 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.430088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d6bec-684d-424d-8c81-54bc82f31096 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.430501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d6bec-684d-424d-8c81-54bc82f31096 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.441077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81588086-1c65-4019-88ef-19e0d38c4d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:45.441770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81588086-1c65-4019-88ef-19e0d38c4d17 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.442050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81588086-1c65-4019-88ef-19e0d38c4d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.442385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81588086-1c65-4019-88ef-19e0d38c4d17 None admin] PATH_INFO: `/v3/users/bb1185ec899e4715af180f73c65d68b8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.443588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81588086-1c65-4019-88ef-19e0d38c4d17 None admin] RBAC: Authorizing `identity:get_user(user_id=bb1185ec899e4715af180f73c65d68b8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:45.445213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d6bec-684d-424d-8c81-54bc82f31096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:45.445581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81588086-1c65-4019-88ef-19e0d38c4d17 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:45.447214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 127/253] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:45 2022] GET /identity/v3/users/bb1185ec899e4715af180f73c65d68b8 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-7b0d6bec-684d-424d-8c81-54bc82f31096 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:45.450600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:45.451677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:24:45.452668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:45.454659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81588086-1c65-4019-88ef-19e0d38c4d17 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:45.454977 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 127/254] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:45.462203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d6bec-684d-424d-8c81-54bc82f31096 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:45.534380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:45.577877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d6bec-684d-424d-8c81-54bc82f31096 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:45.596774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81588086-1c65-4019-88ef-19e0d38c4d17 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:45.626391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff4cee-a5f1-470f-9caa-6730e61630b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:45.629680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff4cee-a5f1-470f-9caa-6730e61630b8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.630159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff4cee-a5f1-470f-9caa-6730e61630b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.630626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff4cee-a5f1-470f-9caa-6730e61630b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.631486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff4cee-a5f1-470f-9caa-6730e61630b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:45.633115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff4cee-a5f1-470f-9caa-6730e61630b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:45.638409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff4cee-a5f1-470f-9caa-6730e61630b8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:45.655004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 128/255] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:45.657405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a242ba-0949-4388-b1bf-43c8a19a63b2 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.657860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a242ba-0949-4388-b1bf-43c8a19a63b2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.658278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a242ba-0949-4388-b1bf-43c8a19a63b2 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.660078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 129/256] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:45.665787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a242ba-0949-4388-b1bf-43c8a19a63b2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:45.667249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25858f22-2b6c-474b-bde0-5af3d77ca9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:45.667901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858f22-2b6c-474b-bde0-5af3d77ca9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.668177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858f22-2b6c-474b-bde0-5af3d77ca9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.668444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858f22-2b6c-474b-bde0-5af3d77ca9d3 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.671442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858f22-2b6c-474b-bde0-5af3d77ca9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:45.671931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858f22-2b6c-474b-bde0-5af3d77ca9d3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:45.677318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 128/257] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:45 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:45.677318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a0f01-5378-4694-81ea-389be754ec9c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.677318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a0f01-5378-4694-81ea-389be754ec9c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.677318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a0f01-5378-4694-81ea-389be754ec9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.698426 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b33a0f01-5378-4694-81ea-389be754ec9c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:45.700069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b33a0f01-5378-4694-81ea-389be754ec9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:45.729694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:45.789512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 129/258] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:45.794023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a0f01-5378-4694-81ea-389be754ec9c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:45.884520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a242ba-0949-4388-b1bf-43c8a19a63b2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:45.911584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:45.927510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8b4f5-a3b4-4e6d-9601-a8eaf754bfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:45.928157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b4f5-a3b4-4e6d-9601-a8eaf754bfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:45.928434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b4f5-a3b4-4e6d-9601-a8eaf754bfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:45.928733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b4f5-a3b4-4e6d-9601-a8eaf754bfc5 None admin] PATH_INFO: `/v3/users/ecff1398f8e14ef7b916c6c3c08928de` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:45.930298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8b4f5-a3b4-4e6d-9601-a8eaf754bfc5 None admin] RBAC: Authorizing `identity:get_user(user_id=ecff1398f8e14ef7b916c6c3c08928de)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:45.931816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8b4f5-a3b4-4e6d-9601-a8eaf754bfc5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:45.933626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 130/259] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:45 2022] GET /identity/v3/users/ecff1398f8e14ef7b916c6c3c08928de => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:45.934631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8b4f5-a3b4-4e6d-9601-a8eaf754bfc5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:46.041456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8b4f5-a3b4-4e6d-9601-a8eaf754bfc5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:46.058714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a0f01-5378-4694-81ea-389be754ec9c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:46.097857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4af9d-621e-4487-8bc3-c260f89be342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:46.098497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4af9d-621e-4487-8bc3-c260f89be342 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:46.098772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4af9d-621e-4487-8bc3-c260f89be342 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:46.099037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4af9d-621e-4487-8bc3-c260f89be342 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:46.120540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4af9d-621e-4487-8bc3-c260f89be342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4fa4af9d-621e-4487-8bc3-c260f89be342 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:46.126669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:46.127534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:46.128339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:46.133251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47ebb-ec62-474e-871c-e65f2518566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:46.133251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47ebb-ec62-474e-871c-e65f2518566e None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:46.133251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47ebb-ec62-474e-871c-e65f2518566e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:46.133251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47ebb-ec62-474e-871c-e65f2518566e None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/users/bb1185ec899e4715af180f73c65d68b8/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:46.133581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a47ebb-ec62-474e-871c-e65f2518566e None admin] RBAC: Authorizing `identity:create_grant(project_id=28871a3f7ee34b03bf66682bb0a217ec, user_id=bb1185ec899e4715af180f73c65d68b8, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:46.134863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a47ebb-ec62-474e-871c-e65f2518566e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:46.142082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 131/260] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:24:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:46.143252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4af9d-621e-4487-8bc3-c260f89be342 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:46.162268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 130/261] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:45 2022] PUT /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/users/bb1185ec899e4715af180f73c65d68b8/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:46.165370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47ebb-ec62-474e-871c-e65f2518566e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:46.216039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:46.290020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:46.406728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4af9d-621e-4487-8bc3-c260f89be342 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:46.408880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47ebb-ec62-474e-871c-e65f2518566e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:46.474095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe52201-c1bd-43f5-8fb5-f1ca102d028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:46.475051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:46.476394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52201-c1bd-43f5-8fb5-f1ca102d028c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:46.477215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:46.477723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:46.478183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52201-c1bd-43f5-8fb5-f1ca102d028c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:46.478641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:46.479095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52201-c1bd-43f5-8fb5-f1ca102d028c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:46.479970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:46.481016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:46.483197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:46.505513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 131/262] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:24:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:46.518344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:46.603457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe52201-c1bd-43f5-8fb5-f1ca102d028c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:46.605139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe52201-c1bd-43f5-8fb5-f1ca102d028c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:46.686338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:46.754067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 132/263] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:46.754580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:46.760611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe52201-c1bd-43f5-8fb5-f1ca102d028c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:46.813668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597facf-169d-49ca-9db2-4e3251a7ac75 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:46.874026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933be05-480e-42e3-a8e1-ee8c3cf22ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:46.875038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933be05-480e-42e3-a8e1-ee8c3cf22ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:46.875509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933be05-480e-42e3-a8e1-ee8c3cf22ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:46.875988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933be05-480e-42e3-a8e1-ee8c3cf22ea2 None admin] PATH_INFO: `/v3/users/ecff1398f8e14ef7b916c6c3c08928de` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:46.877474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933be05-480e-42e3-a8e1-ee8c3cf22ea2 None admin] RBAC: Authorizing `identity:get_user(user_id=ecff1398f8e14ef7b916c6c3c08928de)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:46.879110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933be05-480e-42e3-a8e1-ee8c3cf22ea2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:46.882758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 132/264] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:46 2022] GET /identity/v3/users/ecff1398f8e14ef7b916c6c3c08928de => generated 283 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:46.888932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933be05-480e-42e3-a8e1-ee8c3cf22ea2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:46.981776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe52201-c1bd-43f5-8fb5-f1ca102d028c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:47.049921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b06122-3f19-48ed-8123-d33150af6003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:47.050918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b06122-3f19-48ed-8123-d33150af6003 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.051396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b06122-3f19-48ed-8123-d33150af6003 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.051884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b06122-3f19-48ed-8123-d33150af6003 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.058240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b06122-3f19-48ed-8123-d33150af6003 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:47.059978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b06122-3f19-48ed-8123-d33150af6003 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:47.062035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 133/265] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:46 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:47.070154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b06122-3f19-48ed-8123-d33150af6003 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:47.113877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:47.156267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933be05-480e-42e3-a8e1-ee8c3cf22ea2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:47.235375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b06122-3f19-48ed-8123-d33150af6003 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:47.240111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d82ae1-64a8-4128-87ed-d9ae266c9e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:47.240111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d82ae1-64a8-4128-87ed-d9ae266c9e08 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.240111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d82ae1-64a8-4128-87ed-d9ae266c9e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.240111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d82ae1-64a8-4128-87ed-d9ae266c9e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.280483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:47.317562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:47.318232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.318511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.318782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.320544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec, user.id=ecff1398f8e14ef7b916c6c3c08928de) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:47.320853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:47.322537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:47.334589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 134/266] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:47 2022] GET /identity/v3/role_assignments?user.id=ecff1398f8e14ef7b916c6c3c08928de&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:47.388500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d82ae1-64a8-4128-87ed-d9ae266c9e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:47.390097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d82ae1-64a8-4128-87ed-d9ae266c9e08 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:47.550441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 133/267] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:47.556537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:47.685623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:47.708061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2acc95-9285-4d29-9cef-dff3ffa50248 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:47.744731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597193e2-5417-423f-9dd1-92f2c57324b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:47.748375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597193e2-5417-423f-9dd1-92f2c57324b8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.748696 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597193e2-5417-423f-9dd1-92f2c57324b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.748978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597193e2-5417-423f-9dd1-92f2c57324b8 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.750300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597193e2-5417-423f-9dd1-92f2c57324b8 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:47.751794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597193e2-5417-423f-9dd1-92f2c57324b8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:47.753891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 135/268] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:47 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:47.760971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d82ae1-64a8-4128-87ed-d9ae266c9e08 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:47.813762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd3ff-bbc4-45fc-aaf4-07581215a2f5 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.814050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd3ff-bbc4-45fc-aaf4-07581215a2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.814233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd3ff-bbc4-45fc-aaf4-07581215a2f5 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.816189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 136/269] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:47.821404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:47.822644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db91cb-9ccb-431f-8f6c-7391437fce67 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.822999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db91cb-9ccb-431f-8f6c-7391437fce67 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.823308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db91cb-9ccb-431f-8f6c-7391437fce67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.850826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f5db91cb-9ccb-431f-8f6c-7391437fce67 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:47.852023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f5db91cb-9ccb-431f-8f6c-7391437fce67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:47.880938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d82ae1-64a8-4128-87ed-d9ae266c9e08 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:47.921292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa667836-6326-43c5-a62e-65d9ec8388fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:47.921914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667836-6326-43c5-a62e-65d9ec8388fc None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.922188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667836-6326-43c5-a62e-65d9ec8388fc None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.922456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667836-6326-43c5-a62e-65d9ec8388fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.923190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa667836-6326-43c5-a62e-65d9ec8388fc None admin] RBAC: Adding query filter params (role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec, user.id=ecff1398f8e14ef7b916c6c3c08928de) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:47.923492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa667836-6326-43c5-a62e-65d9ec8388fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:47.925202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa667836-6326-43c5-a62e-65d9ec8388fc None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:47.932585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 134/270] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:47 2022] GET /identity/v3/role_assignments?user.id=ecff1398f8e14ef7b916c6c3c08928de&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:47.934650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a0ed-dab5-4ced-9572-2f4ab882d770 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.934872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a0ed-dab5-4ced-9572-2f4ab882d770 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.935089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a0ed-dab5-4ced-9572-2f4ab882d770 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.941330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 135/271] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:47.946758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e7505-5951-4d31-b391-63d6e97754b5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:47.947049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e7505-5951-4d31-b391-63d6e97754b5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:47.947302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e7505-5951-4d31-b391-63d6e97754b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:47.987473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 137/272] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:47.992220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4d7e7505-5951-4d31-b391-63d6e97754b5 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:47.995391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4d7e7505-5951-4d31-b391-63d6e97754b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:48.022922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db91cb-9ccb-431f-8f6c-7391437fce67 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.092362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 136/273] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:48.125313 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e7505-5951-4d31-b391-63d6e97754b5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.187057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:48.191282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:48.215719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db91cb-9ccb-431f-8f6c-7391437fce67 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:48.275892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4544c-c7eb-4e91-b435-51bc0fc7e5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:48.276881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4544c-c7eb-4e91-b435-51bc0fc7e5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.277375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4544c-c7eb-4e91-b435-51bc0fc7e5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.277863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4544c-c7eb-4e91-b435-51bc0fc7e5f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.292733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e7505-5951-4d31-b391-63d6e97754b5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:48.293485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4544c-c7eb-4e91-b435-51bc0fc7e5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-46e4544c-c7eb-4e91-b435-51bc0fc7e5f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:48.298371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:48.299499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:48.300316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 138/274] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:48.301063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280f33f-6c00-4c13-a82c-cdbc152b0511 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.301063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280f33f-6c00-4c13-a82c-cdbc152b0511 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.301834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280f33f-6c00-4c13-a82c-cdbc152b0511 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.305273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 139/275] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:48.308668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3155-352e-4c88-93bf-0fdbd8d5111c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.309023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3155-352e-4c88-93bf-0fdbd8d5111c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.309327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3155-352e-4c88-93bf-0fdbd8d5111c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.331780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-cdbb3155-352e-4c88-93bf-0fdbd8d5111c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:48.333198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cdbb3155-352e-4c88-93bf-0fdbd8d5111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:48.337025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f9145-935e-42f4-bde0-466867dce8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:48.337754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f9145-935e-42f4-bde0-466867dce8eb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.337990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f9145-935e-42f4-bde0-466867dce8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.338590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f9145-935e-42f4-bde0-466867dce8eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.339373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f9145-935e-42f4-bde0-466867dce8eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:48.340819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f9145-935e-42f4-bde0-466867dce8eb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:48.347269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 137/276] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:48.347269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f9145-935e-42f4-bde0-466867dce8eb None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.413351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 140/277] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:48.413351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb3155-352e-4c88-93bf-0fdbd8d5111c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.480826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f9145-935e-42f4-bde0-466867dce8eb None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:48.492455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:48.502313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:48.519996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:48.520879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.521159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.521744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.522485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:48.522791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:48.524597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:48.533815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 138/278] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:48.537105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb3155-352e-4c88-93bf-0fdbd8d5111c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:48.539667 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.575374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6394bb-edaf-496a-90fe-b3a840c75f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:48.576300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6394bb-edaf-496a-90fe-b3a840c75f58 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.576877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6394bb-edaf-496a-90fe-b3a840c75f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.577383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6394bb-edaf-496a-90fe-b3a840c75f58 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.578314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6394bb-edaf-496a-90fe-b3a840c75f58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:48.579896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6394bb-edaf-496a-90fe-b3a840c75f58 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:48.604786 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 141/279] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:24:48 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:48.614412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6394bb-edaf-496a-90fe-b3a840c75f58 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.660103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4387aa1-614c-4002-bda0-82ee75a23628 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:48.706705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28739197-4956-492b-82d8-2e7986ac99d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:48.707820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28739197-4956-492b-82d8-2e7986ac99d7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.708304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28739197-4956-492b-82d8-2e7986ac99d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.708852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28739197-4956-492b-82d8-2e7986ac99d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.723770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28739197-4956-492b-82d8-2e7986ac99d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-28739197-4956-492b-82d8-2e7986ac99d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:48.730410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:48.731461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:48.732349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:48.735906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 139/280] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:48.741480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6394bb-edaf-496a-90fe-b3a840c75f58 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:48.742034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28739197-4956-492b-82d8-2e7986ac99d7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.774620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:48.778078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:48.790013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a61f61-9110-4070-a99c-16b243d7fa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:48.790665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a61f61-9110-4070-a99c-16b243d7fa30 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.790946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a61f61-9110-4070-a99c-16b243d7fa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.791217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a61f61-9110-4070-a99c-16b243d7fa30 None admin] PATH_INFO: `/v3/users/9ebb340be2444c48beb8239a752a7755` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.811837 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a61f61-9110-4070-a99c-16b243d7fa30 None admin] RBAC: Authorizing `identity:get_user(user_id=9ebb340be2444c48beb8239a752a7755)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:48.813906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a61f61-9110-4070-a99c-16b243d7fa30 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:48.815842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 142/281] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/users/9ebb340be2444c48beb8239a752a7755 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:48.824204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a61f61-9110-4070-a99c-16b243d7fa30 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.876749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28739197-4956-492b-82d8-2e7986ac99d7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:48.922463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e8cac-50c0-4845-8a6e-22180be0fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:48.922957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e8cac-50c0-4845-8a6e-22180be0fe5d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.923229 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e8cac-50c0-4845-8a6e-22180be0fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.923523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e8cac-50c0-4845-8a6e-22180be0fe5d None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.924161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e8cac-50c0-4845-8a6e-22180be0fe5d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:48.924455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e8cac-50c0-4845-8a6e-22180be0fe5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:48.925872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e8cac-50c0-4845-8a6e-22180be0fe5d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:48.943534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 140/282] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:48.945508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830eb3-f6b5-439b-8ebd-70e300bf59b8 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.945714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830eb3-f6b5-439b-8ebd-70e300bf59b8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.945978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830eb3-f6b5-439b-8ebd-70e300bf59b8 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.947334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 141/283] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:48.950306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93830eb3-f6b5-439b-8ebd-70e300bf59b8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.950426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a61f61-9110-4070-a99c-16b243d7fa30 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:49.000829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e61ee-4b1b-4467-900c-30988afdee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:49.002472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e61ee-4b1b-4467-900c-30988afdee8f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:49.002971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e61ee-4b1b-4467-900c-30988afdee8f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:49.003885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e61ee-4b1b-4467-900c-30988afdee8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:49.005349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e61ee-4b1b-4467-900c-30988afdee8f None admin] RBAC: Adding query filter params (role.id=49b24f3e5ebc4b08b538432eda6b4b5d, user.id=9ebb340be2444c48beb8239a752a7755, scope.system=all) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:49.006556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e61ee-4b1b-4467-900c-30988afdee8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:49.010238 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e61ee-4b1b-4467-900c-30988afdee8f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:49.013969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 143/284] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/role_assignments?user.id=9ebb340be2444c48beb8239a752a7755&scope.system=all&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:49.015163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d7e6-4f9f-4dbc-a3dc-eca0f9dcc844 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:49.015440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d7e6-4f9f-4dbc-a3dc-eca0f9dcc844 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:49.015707 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d7e6-4f9f-4dbc-a3dc-eca0f9dcc844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:49.055683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6047d7e6-4f9f-4dbc-a3dc-eca0f9dcc844 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:49.057072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6047d7e6-4f9f-4dbc-a3dc-eca0f9dcc844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:49.072388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:49.084212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93830eb3-f6b5-439b-8ebd-70e300bf59b8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:49.133809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ea770-1dd6-4e69-bbaa-60a40faff1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:49.135113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea770-1dd6-4e69-bbaa-60a40faff1cd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:49.136941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea770-1dd6-4e69-bbaa-60a40faff1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:49.137454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea770-1dd6-4e69-bbaa-60a40faff1cd None admin] PATH_INFO: `/v3/users/61593889fa6142e28a1a49b93bc09908` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:49.141259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ea770-1dd6-4e69-bbaa-60a40faff1cd None admin] RBAC: Authorizing `identity:get_user(user_id=61593889fa6142e28a1a49b93bc09908)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:49.143486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ea770-1dd6-4e69-bbaa-60a40faff1cd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:49.146378 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 142/285] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/users/61593889fa6142e28a1a49b93bc09908 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:49.155073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ea770-1dd6-4e69-bbaa-60a40faff1cd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:49.176888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:49.206465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 144/286] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:49.255219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047d7e6-4f9f-4dbc-a3dc-eca0f9dcc844 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:49.371660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ea770-1dd6-4e69-bbaa-60a40faff1cd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:49.440523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047d7e6-4f9f-4dbc-a3dc-eca0f9dcc844 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:49.443799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189fa91-5f80-46f2-81b4-93243064f64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:49.444483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189fa91-5f80-46f2-81b4-93243064f64d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:49.444756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189fa91-5f80-46f2-81b4-93243064f64d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:49.445022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189fa91-5f80-46f2-81b4-93243064f64d None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:49.447143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189fa91-5f80-46f2-81b4-93243064f64d None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:49.448608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189fa91-5f80-46f2-81b4-93243064f64d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:49.451631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 143/287] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:49 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:49.457333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189fa91-5f80-46f2-81b4-93243064f64d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:49.498995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76297cc-53c5-41e2-b332-d1b05fed2362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:49.499639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76297cc-53c5-41e2-b332-d1b05fed2362 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:49.499900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76297cc-53c5-41e2-b332-d1b05fed2362 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:49.500165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76297cc-53c5-41e2-b332-d1b05fed2362 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:49.511114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76297cc-53c5-41e2-b332-d1b05fed2362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e76297cc-53c5-41e2-b332-d1b05fed2362 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:49.515816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:49.516728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:49.517550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:49.518678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 145/288] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:49.525717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76297cc-53c5-41e2-b332-d1b05fed2362 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:49.527283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:49.630464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:49.640209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189fa91-5f80-46f2-81b4-93243064f64d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:49.730230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea7eb0-e94f-41ac-abf3-8fe344490bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:49.731229 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea7eb0-e94f-41ac-abf3-8fe344490bf6 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:49.731697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea7eb0-e94f-41ac-abf3-8fe344490bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:49.732161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea7eb0-e94f-41ac-abf3-8fe344490bf6 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/users/61593889fa6142e28a1a49b93bc09908/roles/9ee2cd43468941e28f08b0be54868574` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:49.743454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea7eb0-e94f-41ac-abf3-8fe344490bf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=28871a3f7ee34b03bf66682bb0a217ec, user_id=61593889fa6142e28a1a49b93bc09908, role_id=9ee2cd43468941e28f08b0be54868574)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:49.746107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea7eb0-e94f-41ac-abf3-8fe344490bf6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:49.766599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76297cc-53c5-41e2-b332-d1b05fed2362 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:49.779306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 144/289] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:24:49 2022] PUT /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/users/61593889fa6142e28a1a49b93bc09908/roles/9ee2cd43468941e28f08b0be54868574 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:49.834781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:49.835388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:49.835666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:49.835985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:49.836659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:49.836951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:49.838483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:49.855616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 146/290] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:49.864173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea7eb0-e94f-41ac-abf3-8fe344490bf6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:50.056376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:50.068898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea7eb0-e94f-41ac-abf3-8fe344490bf6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:50.117587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ce3da-3311-44fd-8c95-0b7a6baf7ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:50.118228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce3da-3311-44fd-8c95-0b7a6baf7ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:50.118501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce3da-3311-44fd-8c95-0b7a6baf7ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:50.118768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce3da-3311-44fd-8c95-0b7a6baf7ae8 None admin] PATH_INFO: `/v3/users/bb1185ec899e4715af180f73c65d68b8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:50.123308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ce3da-3311-44fd-8c95-0b7a6baf7ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=bb1185ec899e4715af180f73c65d68b8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:50.124765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ce3da-3311-44fd-8c95-0b7a6baf7ae8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:50.126415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 145/291] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:49 2022] GET /identity/v3/users/bb1185ec899e4715af180f73c65d68b8 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:50.131976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:50.261737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ec867-38eb-4cc5-8e77-e39f4b307454 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:50.263346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:50.303123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d5ea-a505-4d12-863b-804cfc00f14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:50.304254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d5ea-a505-4d12-863b-804cfc00f14e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:50.304589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d5ea-a505-4d12-863b-804cfc00f14e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:50.304857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d5ea-a505-4d12-863b-804cfc00f14e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:50.409935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d5ea-a505-4d12-863b-804cfc00f14e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:50.411582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d5ea-a505-4d12-863b-804cfc00f14e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:50.536163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 147/292] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:50.541964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ce3da-3311-44fd-8c95-0b7a6baf7ae8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:50.691528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ce3da-3311-44fd-8c95-0b7a6baf7ae8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:50.740195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f67808-7a87-4cf4-86f9-f0ef1c2eb6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:50.741254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f67808-7a87-4cf4-86f9-f0ef1c2eb6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:50.741733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f67808-7a87-4cf4-86f9-f0ef1c2eb6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:50.742194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f67808-7a87-4cf4-86f9-f0ef1c2eb6a4 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:50.750054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f67808-7a87-4cf4-86f9-f0ef1c2eb6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:50.751773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f67808-7a87-4cf4-86f9-f0ef1c2eb6a4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:50.754290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 146/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:50 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:50.761816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d5ea-a505-4d12-863b-804cfc00f14e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:50.764001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:50.922599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d5ea-a505-4d12-863b-804cfc00f14e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:50.973167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58db8-f410-4835-8f6a-1fc93580a103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:50.973717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58db8-f410-4835-8f6a-1fc93580a103 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:50.973993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58db8-f410-4835-8f6a-1fc93580a103 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:50.974304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58db8-f410-4835-8f6a-1fc93580a103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:50.975032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58db8-f410-4835-8f6a-1fc93580a103 None admin] RBAC: Adding query filter params (role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec, user.id=bb1185ec899e4715af180f73c65d68b8) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:50.975358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58db8-f410-4835-8f6a-1fc93580a103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:50.976848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58db8-f410-4835-8f6a-1fc93580a103 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:50.985989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 148/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:50 2022] GET /identity/v3/role_assignments?user.id=bb1185ec899e4715af180f73c65d68b8&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:51.360629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33948d3-d872-491b-899b-fa00ee5620cb None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.361178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33948d3-d872-491b-899b-fa00ee5620cb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.361178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33948d3-d872-491b-899b-fa00ee5620cb None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.362779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 147/295] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:51.369309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be222-1ef5-4c89-a699-f345eb93cc39 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.369586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be222-1ef5-4c89-a699-f345eb93cc39 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.369849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be222-1ef5-4c89-a699-f345eb93cc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.373512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28627a1f-fbc7-4ef4-abce-3fecf8657635 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.374015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28627a1f-fbc7-4ef4-abce-3fecf8657635 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.374484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28627a1f-fbc7-4ef4-abce-3fecf8657635 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.376084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 148/296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:51.381738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8c2f4-4680-4219-bd80-4010817c7517 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.382302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8c2f4-4680-4219-bd80-4010817c7517 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.382776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8c2f4-4680-4219-bd80-4010817c7517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.390367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-767be222-1ef5-4c89-a699-f345eb93cc39 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:51.391772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-767be222-1ef5-4c89-a699-f345eb93cc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:51.408184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1ad8c2f4-4680-4219-bd80-4010817c7517 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:51.409421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1ad8c2f4-4680-4219-bd80-4010817c7517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:51.438898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:51.497980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:51.519755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 149/297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:51.532405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 149/298] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:51.545756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767be222-1ef5-4c89-a699-f345eb93cc39 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:51.554896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8c2f4-4680-4219-bd80-4010817c7517 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:51.665385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8c2f4-4680-4219-bd80-4010817c7517 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:51.684442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767be222-1ef5-4c89-a699-f345eb93cc39 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:51.704326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da019fc-aad1-422e-ab3f-f26cd3581a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:51.704965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da019fc-aad1-422e-ab3f-f26cd3581a9e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.705254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da019fc-aad1-422e-ab3f-f26cd3581a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.705539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da019fc-aad1-422e-ab3f-f26cd3581a9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.717673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da019fc-aad1-422e-ab3f-f26cd3581a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-8da019fc-aad1-422e-ab3f-f26cd3581a9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:51.722513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:51.723396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:51.724184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:51.728569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 150/299] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:51.729499 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da019fc-aad1-422e-ab3f-f26cd3581a9e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:51.731249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d17ad7-2182-4113-a218-53a9e2e14ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:51.732036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17ad7-2182-4113-a218-53a9e2e14ebc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.732720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17ad7-2182-4113-a218-53a9e2e14ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.733456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17ad7-2182-4113-a218-53a9e2e14ebc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.747963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d17ad7-2182-4113-a218-53a9e2e14ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:51.750007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-74d17ad7-2182-4113-a218-53a9e2e14ebc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:51.752953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:51.753897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:51.754711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:51.758174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 150/300] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:51.762101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c751e-7ca3-4695-9583-b98c30c6956f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.762394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c751e-7ca3-4695-9583-b98c30c6956f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.762849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c751e-7ca3-4695-9583-b98c30c6956f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.764324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 151/301] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:51.769104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c751e-7ca3-4695-9583-b98c30c6956f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:51.840292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:51.851260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da019fc-aad1-422e-ab3f-f26cd3581a9e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:51.891169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c751e-7ca3-4695-9583-b98c30c6956f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:51.908745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce2020-c186-40d5-970f-d3a20135d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:51.909405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2020-c186-40d5-970f-d3a20135d99c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.909686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2020-c186-40d5-970f-d3a20135d99c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.909955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2020-c186-40d5-970f-d3a20135d99c None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.910627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce2020-c186-40d5-970f-d3a20135d99c None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:51.910931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce2020-c186-40d5-970f-d3a20135d99c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:51.912299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce2020-c186-40d5-970f-d3a20135d99c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:51.922055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 151/302] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:24:51.923298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782725f-b145-49e2-bcba-e60b29c7aa05 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.923573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782725f-b145-49e2-bcba-e60b29c7aa05 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.923842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782725f-b145-49e2-bcba-e60b29c7aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.936083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:51.936740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:51.936968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:51.937637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:51.938061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:51.938358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:51.939742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:51.942500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b782725f-b145-49e2-bcba-e60b29c7aa05 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:51.943725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b782725f-b145-49e2-bcba-e60b29c7aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:51.950063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 152/303] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:51.955528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:52.020697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 152/304] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:52.021708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782725f-b145-49e2-bcba-e60b29c7aa05 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:52.043133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:52.085834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c18cc2-5410-4ae1-b831-7e541c9f5d2b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:52.105154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:52.124990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c9f6a-66c7-4e59-a7b4-37cc2943c212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:52.125670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c9f6a-66c7-4e59-a7b4-37cc2943c212 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.126326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c9f6a-66c7-4e59-a7b4-37cc2943c212 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.126605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c9f6a-66c7-4e59-a7b4-37cc2943c212 None admin] PATH_INFO: `/v3/users/9ebb340be2444c48beb8239a752a7755` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.128088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c9f6a-66c7-4e59-a7b4-37cc2943c212 None admin] RBAC: Authorizing `identity:get_user(user_id=9ebb340be2444c48beb8239a752a7755)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:52.130021 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c9f6a-66c7-4e59-a7b4-37cc2943c212 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:52.136558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 153/305] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:51 2022] GET /identity/v3/users/9ebb340be2444c48beb8239a752a7755 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:52.136558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c9f6a-66c7-4e59-a7b4-37cc2943c212 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:52.156174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782725f-b145-49e2-bcba-e60b29c7aa05 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:52.194537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbe036-6469-4053-8875-f99b8d684fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:52.195171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbe036-6469-4053-8875-f99b8d684fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.195454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbe036-6469-4053-8875-f99b8d684fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.195731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbe036-6469-4053-8875-f99b8d684fd3 None admin] PATH_INFO: `/v3/users/65a850c396b24b84844de27566a21279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.213824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbe036-6469-4053-8875-f99b8d684fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=65a850c396b24b84844de27566a21279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:52.215268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbe036-6469-4053-8875-f99b8d684fd3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:52.217076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 153/306] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:52 2022] GET /identity/v3/users/65a850c396b24b84844de27566a21279 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:52.222092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbe036-6469-4053-8875-f99b8d684fd3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:52.246658 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c9f6a-66c7-4e59-a7b4-37cc2943c212 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:52.295164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154f5d3-96d9-4274-94e4-c6584b94004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:52.295789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154f5d3-96d9-4274-94e4-c6584b94004c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.296061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154f5d3-96d9-4274-94e4-c6584b94004c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.296324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154f5d3-96d9-4274-94e4-c6584b94004c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.297449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154f5d3-96d9-4274-94e4-c6584b94004c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:52.298890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154f5d3-96d9-4274-94e4-c6584b94004c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:52.301299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 154/307] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:52.301299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154f5d3-96d9-4274-94e4-c6584b94004c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:52.302035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:52.342904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbe036-6469-4053-8875-f99b8d684fd3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:52.366463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:52.379369 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c799218-f94a-4e5d-9f91-cede64d45cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:52.380222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c799218-f94a-4e5d-9f91-cede64d45cd9 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.380372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c799218-f94a-4e5d-9f91-cede64d45cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.380548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c799218-f94a-4e5d-9f91-cede64d45cd9 None admin] PATH_INFO: `/v3/system/users/9ebb340be2444c48beb8239a752a7755/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.382565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c799218-f94a-4e5d-9f91-cede64d45cd9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9ebb340be2444c48beb8239a752a7755, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:52.383936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c799218-f94a-4e5d-9f91-cede64d45cd9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:52.393070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 154/308] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Aug 10 13:24:52 2022] PUT /identity/v3/system/users/9ebb340be2444c48beb8239a752a7755/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:52.398726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c799218-f94a-4e5d-9f91-cede64d45cd9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:52.455158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154f5d3-96d9-4274-94e4-c6584b94004c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:52.504617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec583d1-15f0-4d93-9984-dddf8d76212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:52.505248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec583d1-15f0-4d93-9984-dddf8d76212b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.505654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec583d1-15f0-4d93-9984-dddf8d76212b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.505922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec583d1-15f0-4d93-9984-dddf8d76212b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.507250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec583d1-15f0-4d93-9984-dddf8d76212b None admin] RBAC: Adding query filter params (role.id=9ee2cd43468941e28f08b0be54868574, user.id=65a850c396b24b84844de27566a21279, scope.system=all) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:52.507551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec583d1-15f0-4d93-9984-dddf8d76212b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:52.509029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec583d1-15f0-4d93-9984-dddf8d76212b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:52.514717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 155/309] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 10 13:24:52 2022] GET /identity/v3/role_assignments?user.id=65a850c396b24b84844de27566a21279&scope.system=all&role.id=9ee2cd43468941e28f08b0be54868574 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:52.525441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a26272-22a5-4781-9586-a0c1b65c9273 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.525441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a26272-22a5-4781-9586-a0c1b65c9273 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.525441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a26272-22a5-4781-9586-a0c1b65c9273 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.525441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 156/310] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:52.525441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b909-240b-4da2-89c2-29b38285f86e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.525441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b909-240b-4da2-89c2-29b38285f86e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.525826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b909-240b-4da2-89c2-29b38285f86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.531447 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c799218-f94a-4e5d-9f91-cede64d45cd9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:52.545638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0ca7b909-240b-4da2-89c2-29b38285f86e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:52.546983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0ca7b909-240b-4da2-89c2-29b38285f86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:52.574650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:52.588422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0f4b0-e156-4686-8560-0cc0bd730b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:52.589113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0f4b0-e156-4686-8560-0cc0bd730b95 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.589426 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0f4b0-e156-4686-8560-0cc0bd730b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.589689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0f4b0-e156-4686-8560-0cc0bd730b95 None admin] PATH_INFO: `/v3/groups` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.590319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0f4b0-e156-4686-8560-0cc0bd730b95 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:52.591980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0f4b0-e156-4686-8560-0cc0bd730b95 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:52.621394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 155/311] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Aug 10 13:24:52 2022] POST /identity/v3/groups => generated 229 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:52.660837 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 157/312] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:52.686265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0f4b0-e156-4686-8560-0cc0bd730b95 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:52.821001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0f4b0-e156-4686-8560-0cc0bd730b95 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:52.872349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657e1d9-f720-443d-9a40-b0cb61762f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:52.873381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657e1d9-f720-443d-9a40-b0cb61762f66 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:52.873885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657e1d9-f720-443d-9a40-b0cb61762f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:52.874363 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657e1d9-f720-443d-9a40-b0cb61762f66 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:52.886469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657e1d9-f720-443d-9a40-b0cb61762f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b657e1d9-f720-443d-9a40-b0cb61762f66 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:52.892114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:52.892994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:52.893808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:52.896452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 156/313] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:52.905285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7b909-240b-4da2-89c2-29b38285f86e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:52.951922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:53.012176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7b909-240b-4da2-89c2-29b38285f86e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:53.050728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:53.051342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:53.051618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:53.051890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:53.052569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:53.052869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:53.054985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:53.064581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 158/314] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:53.070246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657e1d9-f720-443d-9a40-b0cb61762f66 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:53.135803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:53.196748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657e1d9-f720-443d-9a40-b0cb61762f66 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:53.233166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1fa08-3a89-4608-9b90-336ee18aa7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:53.234219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1fa08-3a89-4608-9b90-336ee18aa7ed None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:53.234698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1fa08-3a89-4608-9b90-336ee18aa7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:53.235165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1fa08-3a89-4608-9b90-336ee18aa7ed None admin] PATH_INFO: `/v3/users/61593889fa6142e28a1a49b93bc09908` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:53.236525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1fa08-3a89-4608-9b90-336ee18aa7ed None admin] RBAC: Authorizing `identity:get_user(user_id=61593889fa6142e28a1a49b93bc09908)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:53.238223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1fa08-3a89-4608-9b90-336ee18aa7ed None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:53.240389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 157/315] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:53 2022] GET /identity/v3/users/61593889fa6142e28a1a49b93bc09908 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:53.249295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:53.364767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce294-0727-4478-8c71-77299f04f81c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:53.375560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:53.400571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf381e-92f2-43c1-b94c-bd7a170aef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:53.401094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf381e-92f2-43c1-b94c-bd7a170aef9e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:53.401403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf381e-92f2-43c1-b94c-bd7a170aef9e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:53.401666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf381e-92f2-43c1-b94c-bd7a170aef9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:53.481093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf381e-92f2-43c1-b94c-bd7a170aef9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:53.483983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf381e-92f2-43c1-b94c-bd7a170aef9e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:53.606697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 159/316] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:53.614314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1fa08-3a89-4608-9b90-336ee18aa7ed None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:53.722025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1fa08-3a89-4608-9b90-336ee18aa7ed None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:53.739721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:53.757829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e42f6-feec-4958-a260-69f105b8a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:53.758464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e42f6-feec-4958-a260-69f105b8a02a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:53.758745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e42f6-feec-4958-a260-69f105b8a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:53.759010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e42f6-feec-4958-a260-69f105b8a02a None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:53.760177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e42f6-feec-4958-a260-69f105b8a02a None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:53.761679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e42f6-feec-4958-a260-69f105b8a02a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:53.763520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 158/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:53 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:53.769259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf381e-92f2-43c1-b94c-bd7a170aef9e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:53.781590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:53.883582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf381e-92f2-43c1-b94c-bd7a170aef9e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:53.929937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d5f93-aad2-4b03-b70b-096d83036d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:53.930617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d5f93-aad2-4b03-b70b-096d83036d15 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:53.930891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d5f93-aad2-4b03-b70b-096d83036d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:53.931302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d5f93-aad2-4b03-b70b-096d83036d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:53.932456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d5f93-aad2-4b03-b70b-096d83036d15 None admin] RBAC: Adding query filter params (role.id=9ee2cd43468941e28f08b0be54868574, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec, user.id=61593889fa6142e28a1a49b93bc09908) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:53.932916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d5f93-aad2-4b03-b70b-096d83036d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:53.935338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d5f93-aad2-4b03-b70b-096d83036d15 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:53.945624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 160/318] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:24:53 2022] GET /identity/v3/role_assignments?user.id=61593889fa6142e28a1a49b93bc09908&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=9ee2cd43468941e28f08b0be54868574 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:54.615832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b8e3-7eb9-4a3f-9827-c8d25f1b6e52 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:54.616337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b8e3-7eb9-4a3f-9827-c8d25f1b6e52 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:54.616337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b8e3-7eb9-4a3f-9827-c8d25f1b6e52 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:54.617707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 159/319] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:54.622775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172f115-37c1-4044-b50a-7da0f65dc3da None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:54.623175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172f115-37c1-4044-b50a-7da0f65dc3da None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:54.623335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172f115-37c1-4044-b50a-7da0f65dc3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:54.628275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dd6fe-07ca-4992-b41b-c304279803fa None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:54.628562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dd6fe-07ca-4992-b41b-c304279803fa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:54.628834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dd6fe-07ca-4992-b41b-c304279803fa None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:54.630222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 160/320] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:24:54.635729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377e53-0265-4a79-b1ff-304fa8c4a326 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:54.636008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377e53-0265-4a79-b1ff-304fa8c4a326 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:54.636285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377e53-0265-4a79-b1ff-304fa8c4a326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:54.641839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8172f115-37c1-4044-b50a-7da0f65dc3da None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:54.643897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8172f115-37c1-4044-b50a-7da0f65dc3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:54.654401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a8377e53-0265-4a79-b1ff-304fa8c4a326 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:54.655582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a8377e53-0265-4a79-b1ff-304fa8c4a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:54.717273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 161/321] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:54.730874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 161/322] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:54.737926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172f115-37c1-4044-b50a-7da0f65dc3da None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:54.741343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:54.753393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8377e53-0265-4a79-b1ff-304fa8c4a326 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:54.843436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172f115-37c1-4044-b50a-7da0f65dc3da None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:54.860907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8377e53-0265-4a79-b1ff-304fa8c4a326 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:54.877631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6d2e1-ebfd-4a91-b451-cf16c512e2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:54.877807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:54.878257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6d2e1-ebfd-4a91-b451-cf16c512e2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:54.878630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6d2e1-ebfd-4a91-b451-cf16c512e2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:54.878826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6d2e1-ebfd-4a91-b451-cf16c512e2c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:54.890468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6d2e1-ebfd-4a91-b451-cf16c512e2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f3e6d2e1-ebfd-4a91-b451-cf16c512e2c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:54.895165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:54.896077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:54.897068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 162/323] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:54.897764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02277-dddf-4cf5-b294-989879b163dc None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:54.897764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02277-dddf-4cf5-b294-989879b163dc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:54.898213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02277-dddf-4cf5-b294-989879b163dc None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:54.901262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 163/324] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:54.904196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7310e3-01e5-4644-bcd4-40e3be95868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:54.904650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b967bd-1bcc-45b5-aa6c-cf0a5ae3ad71 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:54.904799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7310e3-01e5-4644-bcd4-40e3be95868d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:54.904929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b967bd-1bcc-45b5-aa6c-cf0a5ae3ad71 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:54.905085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7310e3-01e5-4644-bcd4-40e3be95868d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:54.905219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b967bd-1bcc-45b5-aa6c-cf0a5ae3ad71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:54.905416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7310e3-01e5-4644-bcd4-40e3be95868d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:54.916107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7310e3-01e5-4644-bcd4-40e3be95868d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-dd7310e3-01e5-4644-bcd4-40e3be95868d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:54.920998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:54.921906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:54.922664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 162/325] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:54.923293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-41b967bd-1bcc-45b5-aa6c-cf0a5ae3ad71 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:54.923370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7310e3-01e5-4644-bcd4-40e3be95868d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:54.924211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-41b967bd-1bcc-45b5-aa6c-cf0a5ae3ad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:54.986549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:54.995050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 164/326] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:24:54.999752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b967bd-1bcc-45b5-aa6c-cf0a5ae3ad71 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:55.035655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7310e3-01e5-4644-bcd4-40e3be95868d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:55.077174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:55.078274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:55.078740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:55.079069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:55.080063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:55.080366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:55.081795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:55.099484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 163/327] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:55.099484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:55.104574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b967bd-1bcc-45b5-aa6c-cf0a5ae3ad71 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:55.139709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:55.140219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:55.140495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:55.140762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:55.141448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:55.141745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:55.143127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:55.144315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:55.152284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 165/328] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:55.153113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:55.213258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:55.215166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021082fa-b46e-44c2-97f8-684859c5fc49 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:55.251031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fc882-1f0d-4155-bcb9-638cdd5f3356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:55.251658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fc882-1f0d-4155-bcb9-638cdd5f3356 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:55.251938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fc882-1f0d-4155-bcb9-638cdd5f3356 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:55.252209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fc882-1f0d-4155-bcb9-638cdd5f3356 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:55.253348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fc882-1f0d-4155-bcb9-638cdd5f3356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:55.254796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fc882-1f0d-4155-bcb9-638cdd5f3356 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:55.256388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 164/329] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:24:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:55.257177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fc882-1f0d-4155-bcb9-638cdd5f3356 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:55.262259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19708bf-cc78-49b8-bd8a-fe7de0a032e7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:55.301814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b6051-d503-4acd-b359-f4a509a97d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:55.302451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b6051-d503-4acd-b359-f4a509a97d1e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:55.302728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b6051-d503-4acd-b359-f4a509a97d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:55.302995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b6051-d503-4acd-b359-f4a509a97d1e None admin] PATH_INFO: `/v3/users/9ebb340be2444c48beb8239a752a7755` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:55.304750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b6051-d503-4acd-b359-f4a509a97d1e None admin] RBAC: Authorizing `identity:get_user(user_id=9ebb340be2444c48beb8239a752a7755)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:55.306237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b6051-d503-4acd-b359-f4a509a97d1e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:55.307828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 166/330] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:55 2022] GET /identity/v3/users/9ebb340be2444c48beb8239a752a7755 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:55.308565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b6051-d503-4acd-b359-f4a509a97d1e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:55.357737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fc882-1f0d-4155-bcb9-638cdd5f3356 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:55.365374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:55.393132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfeef4-572d-4476-aa3c-c4bae4498f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:55.393711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfeef4-572d-4476-aa3c-c4bae4498f0d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:55.393980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfeef4-572d-4476-aa3c-c4bae4498f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:55.394246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfeef4-572d-4476-aa3c-c4bae4498f0d None admin] PATH_INFO: `/v3/users/65a850c396b24b84844de27566a21279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:55.395437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfeef4-572d-4476-aa3c-c4bae4498f0d None admin] RBAC: Authorizing `identity:get_user(user_id=65a850c396b24b84844de27566a21279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:55.396858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfeef4-572d-4476-aa3c-c4bae4498f0d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:55.398457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 165/331] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:55 2022] GET /identity/v3/users/65a850c396b24b84844de27566a21279 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:55.399202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfeef4-572d-4476-aa3c-c4bae4498f0d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:55.410285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b6051-d503-4acd-b359-f4a509a97d1e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:55.434430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:55.445719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e772dc-b9ff-403f-96a1-02f2ca9aaacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:55.446339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e772dc-b9ff-403f-96a1-02f2ca9aaacd None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:55.446600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e772dc-b9ff-403f-96a1-02f2ca9aaacd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:55.446869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e772dc-b9ff-403f-96a1-02f2ca9aaacd None admin] PATH_INFO: `/v3/groups` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:55.447491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e772dc-b9ff-403f-96a1-02f2ca9aaacd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:55.448934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e772dc-b9ff-403f-96a1-02f2ca9aaacd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:55.466714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 167/332] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Aug 10 13:24:55 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:55.467496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e772dc-b9ff-403f-96a1-02f2ca9aaacd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:55.521369 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfeef4-572d-4476-aa3c-c4bae4498f0d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:55.564098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceb7a6-bdbb-4344-b457-58e9f7481558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:55.564743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb7a6-bdbb-4344-b457-58e9f7481558 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:55.565006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb7a6-bdbb-4344-b457-58e9f7481558 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:55.565302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb7a6-bdbb-4344-b457-58e9f7481558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:55.566095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ceb7a6-bdbb-4344-b457-58e9f7481558 None admin] RBAC: Adding query filter params (role.id=49b24f3e5ebc4b08b538432eda6b4b5d, user.id=9ebb340be2444c48beb8239a752a7755, scope.system=all) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:55.566415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ceb7a6-bdbb-4344-b457-58e9f7481558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:55.567935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ceb7a6-bdbb-4344-b457-58e9f7481558 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:55.591552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e772dc-b9ff-403f-96a1-02f2ca9aaacd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:55.593437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 166/333] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 10 13:24:55 2022] GET /identity/v3/role_assignments?user.id=9ebb340be2444c48beb8239a752a7755&scope.system=all&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:55.632894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b500a9b-83b4-493b-accc-4a41c3e11c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:55.633631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500a9b-83b4-493b-accc-4a41c3e11c3e None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:55.633887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500a9b-83b4-493b-accc-4a41c3e11c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:55.634164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500a9b-83b4-493b-accc-4a41c3e11c3e None admin] PATH_INFO: `/v3/system/users/65a850c396b24b84844de27566a21279/roles/9ee2cd43468941e28f08b0be54868574` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:55.635739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b500a9b-83b4-493b-accc-4a41c3e11c3e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=65a850c396b24b84844de27566a21279, role_id=9ee2cd43468941e28f08b0be54868574)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:55.637190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b500a9b-83b4-493b-accc-4a41c3e11c3e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:55.648093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 168/334] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Aug 10 13:24:55 2022] PUT /identity/v3/system/users/65a850c396b24b84844de27566a21279/roles/9ee2cd43468941e28f08b0be54868574 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:57.894050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f41d-f0fd-4145-8bd5-241c15ad7b24 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:57.894050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f41d-f0fd-4145-8bd5-241c15ad7b24 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:57.894050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f41d-f0fd-4145-8bd5-241c15ad7b24 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:57.895235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 167/335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:57.900981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89af09-53c1-4bad-9699-79d3d30cad56 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:57.901455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89af09-53c1-4bad-9699-79d3d30cad56 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:57.901545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89af09-53c1-4bad-9699-79d3d30cad56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:57.925272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7e89af09-53c1-4bad-9699-79d3d30cad56 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:57.926456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7e89af09-53c1-4bad-9699-79d3d30cad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:57.983795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:58.014821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 169/336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:58.030571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08752e61-5520-45c3-a844-c7d92c6fff21 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.031227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08752e61-5520-45c3-a844-c7d92c6fff21 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.031669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08752e61-5520-45c3-a844-c7d92c6fff21 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.034410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 168/337] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:58.040592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174975d-787e-4b1c-aef9-a4cd263d2354 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.040875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174975d-787e-4b1c-aef9-a4cd263d2354 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.041141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174975d-787e-4b1c-aef9-a4cd263d2354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.046363 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08752e61-5520-45c3-a844-c7d92c6fff21 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:58.071519 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0174975d-787e-4b1c-aef9-a4cd263d2354 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:58.072686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0174975d-787e-4b1c-aef9-a4cd263d2354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:58.109872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:58.225715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 170/338] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:58.243100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08752e61-5520-45c3-a844-c7d92c6fff21 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:58.286683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174975d-787e-4b1c-aef9-a4cd263d2354 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:58.308347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8698365-ea7a-4da3-87c4-49b94991d0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:58.308992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8698365-ea7a-4da3-87c4-49b94991d0fe None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.309278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8698365-ea7a-4da3-87c4-49b94991d0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.309545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8698365-ea7a-4da3-87c4-49b94991d0fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.333591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8698365-ea7a-4da3-87c4-49b94991d0fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-c8698365-ea7a-4da3-87c4-49b94991d0fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:58.342930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:58.344030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:58.344809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 169/339] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:58.354261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8698365-ea7a-4da3-87c4-49b94991d0fe None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:58.512054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:58.512223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174975d-787e-4b1c-aef9-a4cd263d2354 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:58.527087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8698365-ea7a-4da3-87c4-49b94991d0fe None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:58.531513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:58.576787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21b98d-1c35-4a47-9468-330823c9938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:58.577446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b98d-1c35-4a47-9468-330823c9938b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.577720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b98d-1c35-4a47-9468-330823c9938b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.577990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b98d-1c35-4a47-9468-330823c9938b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.578670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b98d-1c35-4a47-9468-330823c9938b None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:58.578966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b98d-1c35-4a47-9468-330823c9938b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:58.580348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b98d-1c35-4a47-9468-330823c9938b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:58.589928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 170/340] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:58.591467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe13e4a-7b5b-4a47-af83-163927a4c7e5 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.591890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe13e4a-7b5b-4a47-af83-163927a4c7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.592326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe13e4a-7b5b-4a47-af83-163927a4c7e5 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.593723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 171/341] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:58.597279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b85ed-d9e6-4bfa-85dd-53ff65497435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:58.597279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b85ed-d9e6-4bfa-85dd-53ff65497435 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.597279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b85ed-d9e6-4bfa-85dd-53ff65497435 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.597279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b85ed-d9e6-4bfa-85dd-53ff65497435 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.598525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe13e4a-7b5b-4a47-af83-163927a4c7e5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:58.615962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b85ed-d9e6-4bfa-85dd-53ff65497435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-046b85ed-d9e6-4bfa-85dd-53ff65497435 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:58.629718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:58.630851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 13:24:58.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:58.632411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 171/342] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:24:58.634448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b02d9-cb79-4918-9213-f6a4ba1fa764 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.634792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b02d9-cb79-4918-9213-f6a4ba1fa764 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.635122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b02d9-cb79-4918-9213-f6a4ba1fa764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.661989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2b7b02d9-cb79-4918-9213-f6a4ba1fa764 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:58.663804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2b7b02d9-cb79-4918-9213-f6a4ba1fa764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:58.750254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe13e4a-7b5b-4a47-af83-163927a4c7e5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:58.760918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 172/343] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:58.762716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59381209-f95a-45be-95b1-87699cba4810 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.763261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59381209-f95a-45be-95b1-87699cba4810 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.763916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59381209-f95a-45be-95b1-87699cba4810 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.765685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 173/344] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:24:58.766644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59381209-f95a-45be-95b1-87699cba4810 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:58.795818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8201062-5f94-4cf0-8d22-37882660f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:58.796684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8201062-5f94-4cf0-8d22-37882660f230 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.797172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8201062-5f94-4cf0-8d22-37882660f230 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.797602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8201062-5f94-4cf0-8d22-37882660f230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.838166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:58.911112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8201062-5f94-4cf0-8d22-37882660f230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:58.914195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8201062-5f94-4cf0-8d22-37882660f230 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:58.930993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:58.940695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59381209-f95a-45be-95b1-87699cba4810 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:58.979302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fb9df-d013-42a6-bcb1-4da2684c5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:58.980059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fb9df-d013-42a6-bcb1-4da2684c5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:58.980426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fb9df-d013-42a6-bcb1-4da2684c5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:58.981032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fb9df-d013-42a6-bcb1-4da2684c5ef2 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:58.982220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433fb9df-d013-42a6-bcb1-4da2684c5ef2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:58.982967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433fb9df-d013-42a6-bcb1-4da2684c5ef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:58.984545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433fb9df-d013-42a6-bcb1-4da2684c5ef2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:58.996020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 174/345] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:59.000752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eead2-968e-4dc9-ae1d-61d905e4ca5c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.000752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eead2-968e-4dc9-ae1d-61d905e4ca5c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.000752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eead2-968e-4dc9-ae1d-61d905e4ca5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.027637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-943eead2-968e-4dc9-ae1d-61d905e4ca5c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:59.028829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-943eead2-968e-4dc9-ae1d-61d905e4ca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:59.064879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 172/346] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:24:59.066780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8201062-5f94-4cf0-8d22-37882660f230 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.119159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 175/347] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:24:59.120317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eead2-968e-4dc9-ae1d-61d905e4ca5c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.132043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:59.181590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8201062-5f94-4cf0-8d22-37882660f230 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:59.220304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eebdf-499d-4954-afff-46115817ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:59.221392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eebdf-499d-4954-afff-46115817ddec None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.221868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eebdf-499d-4954-afff-46115817ddec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.222351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eebdf-499d-4954-afff-46115817ddec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.234156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05eebdf-499d-4954-afff-46115817ddec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f05eebdf-499d-4954-afff-46115817ddec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:59.239150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:59.240073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:24:59.240985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:24:59.242144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:59.243851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 173/348] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:59.245049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eebdf-499d-4954-afff-46115817ddec None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.256902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eead2-968e-4dc9-ae1d-61d905e4ca5c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:59.300208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8c4c9-2fd7-4537-b9c5-170a3ddfa60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:59.301403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8c4c9-2fd7-4537-b9c5-170a3ddfa60d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.302012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8c4c9-2fd7-4537-b9c5-170a3ddfa60d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.302496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8c4c9-2fd7-4537-b9c5-170a3ddfa60d None admin] PATH_INFO: `/v3/users/65a850c396b24b84844de27566a21279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.304798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8c4c9-2fd7-4537-b9c5-170a3ddfa60d None admin] RBAC: Authorizing `identity:get_user(user_id=65a850c396b24b84844de27566a21279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:59.306650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8c4c9-2fd7-4537-b9c5-170a3ddfa60d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:59.308728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 176/349] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/users/65a850c396b24b84844de27566a21279 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:59.309927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8c4c9-2fd7-4537-b9c5-170a3ddfa60d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.396490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eebdf-499d-4954-afff-46115817ddec None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:59.421092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:59.445431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b585-8f4a-40d7-aa51-313de19afb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:59.446587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b585-8f4a-40d7-aa51-313de19afb9a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.446587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b585-8f4a-40d7-aa51-313de19afb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.446587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b585-8f4a-40d7-aa51-313de19afb9a None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.447719 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36b585-8f4a-40d7-aa51-313de19afb9a None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:59.449157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36b585-8f4a-40d7-aa51-313de19afb9a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:59.453293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 174/350] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:59.454018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b585-8f4a-40d7-aa51-313de19afb9a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.492279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8c4c9-2fd7-4537-b9c5-170a3ddfa60d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:59.533796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5518fc-5462-43b9-ac65-a38599a5c139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:59.534860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5518fc-5462-43b9-ac65-a38599a5c139 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.535335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5518fc-5462-43b9-ac65-a38599a5c139 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.535860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5518fc-5462-43b9-ac65-a38599a5c139 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.548274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5518fc-5462-43b9-ac65-a38599a5c139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bf5518fc-5462-43b9-ac65-a38599a5c139 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:24:59.556800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:24:59.558163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:24:59.559257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:24:59.560153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b585-8f4a-40d7-aa51-313de19afb9a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:59.562532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 177/351] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:24:59.563743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5518fc-5462-43b9-ac65-a38599a5c139 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.590457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:59.596789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:59.597624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.597768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.598019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.598820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:59.599120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:59.601120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:59.610491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 175/352] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:59.611417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.662282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:59.720421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5518fc-5462-43b9-ac65-a38599a5c139 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:59.723205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71f32d-11aa-45d6-9456-97f90a9b4d99 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:59.759012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef875b-3771-412d-a355-cff7ebd8f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:59.759941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef875b-3771-412d-a355-cff7ebd8f963 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.759941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef875b-3771-412d-a355-cff7ebd8f963 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.760217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef875b-3771-412d-a355-cff7ebd8f963 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.766938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef875b-3771-412d-a355-cff7ebd8f963 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:59.768009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef875b-3771-412d-a355-cff7ebd8f963 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:59.769704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 176/353] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:59.770598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef875b-3771-412d-a355-cff7ebd8f963 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.795743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:59.797366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.798285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.799185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.800872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] RBAC: Adding query filter params (role.id=9ee2cd43468941e28f08b0be54868574, user.id=65a850c396b24b84844de27566a21279, scope.system=all) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:59.801982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:59.805395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:59.826395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 178/354] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/role_assignments?user.id=65a850c396b24b84844de27566a21279&scope.system=all&role.id=9ee2cd43468941e28f08b0be54868574 => generated 510 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:59.827872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.902973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef875b-3771-412d-a355-cff7ebd8f963 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:59.914164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:59.946059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:59.956495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:59.957204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:59.957720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:59.958052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:59.959021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:24:59.959399 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:59.961399 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:59.971688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 177/355] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:24:59.972789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:59.990168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227bd17-7700-4130-aa0a-f735d592468b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:00.034010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95ee33-61b6-43a5-8a42-7f0ed7417a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:00.035256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95ee33-61b6-43a5-8a42-7f0ed7417a6b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.035815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95ee33-61b6-43a5-8a42-7f0ed7417a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.036376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95ee33-61b6-43a5-8a42-7f0ed7417a6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.120352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c3247-5a45-416a-824c-6230571fecad None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:00.141549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95ee33-61b6-43a5-8a42-7f0ed7417a6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:00.143451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95ee33-61b6-43a5-8a42-7f0ed7417a6b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:00.169204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc378a-516d-4f8f-b412-c4e5f0c21c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:00.169777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc378a-516d-4f8f-b412-c4e5f0c21c97 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.170055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc378a-516d-4f8f-b412-c4e5f0c21c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.170322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc378a-516d-4f8f-b412-c4e5f0c21c97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.171062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc378a-516d-4f8f-b412-c4e5f0c21c97 None admin] RBAC: Adding query filter params (group.id=71054dec6b5c40f185e534fbe0c143f8, role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=8d1b82d0c0604e3a98c466c903e60279) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:00.171369 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc378a-516d-4f8f-b412-c4e5f0c21c97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:00.172929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc378a-516d-4f8f-b412-c4e5f0c21c97 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:00.179759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 178/356] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/role_assignments?group.id=71054dec6b5c40f185e534fbe0c143f8&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:00.180881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada02869-a363-4eb3-b9ac-259282ed5bb1 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.181166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada02869-a363-4eb3-b9ac-259282ed5bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.181557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada02869-a363-4eb3-b9ac-259282ed5bb1 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.183039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 179/357] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:00.183845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada02869-a363-4eb3-b9ac-259282ed5bb1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:00.227835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:00.281771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:00.325273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 179/358] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:00.326985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c346-da0e-48ed-ba06-9b1074868059 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.327479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c346-da0e-48ed-ba06-9b1074868059 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.327952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c346-da0e-48ed-ba06-9b1074868059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.351085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c7f7c346-da0e-48ed-ba06-9b1074868059 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:00.353624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c7f7c346-da0e-48ed-ba06-9b1074868059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:00.369669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada02869-a363-4eb3-b9ac-259282ed5bb1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:00.417629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d0368-6e50-4e29-bd48-ab9e8abefcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:00.420508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d0368-6e50-4e29-bd48-ab9e8abefcdd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.420508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d0368-6e50-4e29-bd48-ab9e8abefcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.420508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d0368-6e50-4e29-bd48-ab9e8abefcdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.446699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 180/359] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:00.447572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7c346-da0e-48ed-ba06-9b1074868059 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:00.517830 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d0368-6e50-4e29-bd48-ab9e8abefcdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:00.519285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d0368-6e50-4e29-bd48-ab9e8abefcdd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:00.562982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:00.611128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7c346-da0e-48ed-ba06-9b1074868059 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:00.619661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:00.649263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 180/360] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:00.652274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531c97-1c5e-47d3-9d4f-ce9a820dde9a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.652553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531c97-1c5e-47d3-9d4f-ce9a820dde9a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.652825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531c97-1c5e-47d3-9d4f-ce9a820dde9a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.658009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 181/361] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:00.658009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531c97-1c5e-47d3-9d4f-ce9a820dde9a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:00.666003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954e914-1a48-4415-a48f-f421647c55f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:00.666875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954e914-1a48-4415-a48f-f421647c55f5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.667075 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954e914-1a48-4415-a48f-f421647c55f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.667320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954e914-1a48-4415-a48f-f421647c55f5 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.668794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954e914-1a48-4415-a48f-f421647c55f5 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:00.671135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954e914-1a48-4415-a48f-f421647c55f5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:00.673340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 181/362] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:00 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:00.673340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954e914-1a48-4415-a48f-f421647c55f5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:00.804213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531c97-1c5e-47d3-9d4f-ce9a820dde9a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:00.823644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954e914-1a48-4415-a48f-f421647c55f5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:00.848419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fdcde-47a6-4680-be68-e0375511dffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:00.849598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdcde-47a6-4680-be68-e0375511dffb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.849878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdcde-47a6-4680-be68-e0375511dffb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.850212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdcde-47a6-4680-be68-e0375511dffb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.856470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:00.870845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fdcde-47a6-4680-be68-e0375511dffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:00.880360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c278eb-e02e-49c2-aefb-6f032208dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:00.880583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c278eb-e02e-49c2-aefb-6f032208dd80 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.880802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c278eb-e02e-49c2-aefb-6f032208dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.880964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c278eb-e02e-49c2-aefb-6f032208dd80 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.882860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c278eb-e02e-49c2-aefb-6f032208dd80 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-a74fdcde-47a6-4680-be68-e0375511dffb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:00.883169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:00.884058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:00.884882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:00.885601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c278eb-e02e-49c2-aefb-6f032208dd80 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:00.889300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 182/363] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:00.889447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 182/364] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:00 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:00.889538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe40f7-9017-4b9a-b1ad-387bf08687a9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:00.889538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe40f7-9017-4b9a-b1ad-387bf08687a9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:00.889538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe40f7-9017-4b9a-b1ad-387bf08687a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:00.889723 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c278eb-e02e-49c2-aefb-6f032208dd80 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:00.913162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4afe40f7-9017-4b9a-b1ad-387bf08687a9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:00.914508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4afe40f7-9017-4b9a-b1ad-387bf08687a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:00.948121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:01.011318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 183/365] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:01.013132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe40f7-9017-4b9a-b1ad-387bf08687a9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:01.021478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c278eb-e02e-49c2-aefb-6f032208dd80 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:01.080065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad7f8-b7c0-4a94-b530-cece2f102f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:01.081089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad7f8-b7c0-4a94-b530-cece2f102f71 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:01.081645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad7f8-b7c0-4a94-b530-cece2f102f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:01.081916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad7f8-b7c0-4a94-b530-cece2f102f71 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:01.083070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bad7f8-b7c0-4a94-b530-cece2f102f71 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:01.084615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bad7f8-b7c0-4a94-b530-cece2f102f71 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:01.086430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 183/366] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:00 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:01.087040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad7f8-b7c0-4a94-b530-cece2f102f71 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:01.151900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe40f7-9017-4b9a-b1ad-387bf08687a9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:01.183409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:01.196640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7857f-6580-4744-9e4f-d8bdd18bbcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:01.197307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7857f-6580-4744-9e4f-d8bdd18bbcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:01.197638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7857f-6580-4744-9e4f-d8bdd18bbcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:01.197902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7857f-6580-4744-9e4f-d8bdd18bbcd7 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:01.199518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7857f-6580-4744-9e4f-d8bdd18bbcd7 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:01.201267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7857f-6580-4744-9e4f-d8bdd18bbcd7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:01.202541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 184/367] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:01.203300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7857f-6580-4744-9e4f-d8bdd18bbcd7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:01.224198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad7f8-b7c0-4a94-b530-cece2f102f71 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:01.238261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:01.278312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:01.281181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:01.281181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:01.281181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:01.281181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:01.281719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:01.284155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:01.296639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 184/368] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:01.298011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:01.328857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7857f-6580-4744-9e4f-d8bdd18bbcd7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:01.376324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213e4f2-f174-495b-ade9-5a085fa8fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:01.377379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213e4f2-f174-495b-ade9-5a085fa8fd5a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:01.377936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213e4f2-f174-495b-ade9-5a085fa8fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:01.378215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213e4f2-f174-495b-ade9-5a085fa8fd5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:01.390887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213e4f2-f174-495b-ade9-5a085fa8fd5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-8213e4f2-f174-495b-ade9-5a085fa8fd5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:01.396241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:01.397737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:01.398535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:01.400280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 185/369] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:01.401351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213e4f2-f174-495b-ade9-5a085fa8fd5a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:01.441197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cca48d-a263-47e2-8794-9e014454662d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:01.462609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:01.490251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:01.491385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:01.491657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:01.491921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:01.492643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] RBAC: Adding query filter params (group.id=71054dec6b5c40f185e534fbe0c143f8, role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:01.492939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:01.494418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:01.500331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 185/370] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/role_assignments?group.id=71054dec6b5c40f185e534fbe0c143f8&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:01.502176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:01.527405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213e4f2-f174-495b-ade9-5a085fa8fd5a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:01.545772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:01.574096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:01.574725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:01.575005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:01.575275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:01.576010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] RBAC: Adding query filter params (group.id=71054dec6b5c40f185e534fbe0c143f8, role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=8d1b82d0c0604e3a98c466c903e60279) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:01.576786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:01.578056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:01.586352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 186/371] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/role_assignments?group.id=71054dec6b5c40f185e534fbe0c143f8&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:01.587048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:01.649059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b41751-7316-49b4-b71f-6114e56597d1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:01.698176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd94cd1-a72f-41ed-a067-3de89cb12b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:01.699159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd94cd1-a72f-41ed-a067-3de89cb12b41 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:01.699593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd94cd1-a72f-41ed-a067-3de89cb12b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:01.700642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd94cd1-a72f-41ed-a067-3de89cb12b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:01.729919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d798236-7e82-40dc-8773-5efdb99a74be None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:01.756068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:01.786745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:01.787833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:01.788262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:01.788988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:01.790720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:01.791190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:01.793652 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:01.805637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd94cd1-a72f-41ed-a067-3de89cb12b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:01.807148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd94cd1-a72f-41ed-a067-3de89cb12b41 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:01.811394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 187/372] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:01.818226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:01.875851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:01.966325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 186/373] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:01.971457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd94cd1-a72f-41ed-a067-3de89cb12b41 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:02.026674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614fd0b-c797-436e-aadf-69abe3fcad74 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:02.103075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb939fb7-2596-4661-a9f3-014a3cd189b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:02.103701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939fb7-2596-4661-a9f3-014a3cd189b8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.103978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939fb7-2596-4661-a9f3-014a3cd189b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.104262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939fb7-2596-4661-a9f3-014a3cd189b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.107205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd94cd1-a72f-41ed-a067-3de89cb12b41 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:02.170703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4b2e-0fc9-46ef-9b7b-1b8c2675a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:02.171392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4b2e-0fc9-46ef-9b7b-1b8c2675a01c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.171675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4b2e-0fc9-46ef-9b7b-1b8c2675a01c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.172053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4b2e-0fc9-46ef-9b7b-1b8c2675a01c None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.174990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4b2e-0fc9-46ef-9b7b-1b8c2675a01c None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:02.176467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4b2e-0fc9-46ef-9b7b-1b8c2675a01c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:02.179794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 187/374] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:02.184460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:02.185265 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4b2e-0fc9-46ef-9b7b-1b8c2675a01c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:02.197504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:02.230443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb939fb7-2596-4661-a9f3-014a3cd189b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:02.231920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb939fb7-2596-4661-a9f3-014a3cd189b8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:02.337250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4b2e-0fc9-46ef-9b7b-1b8c2675a01c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:02.376856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f94ff-ecf9-431c-82ab-88d0992cf8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:02.377882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f94ff-ecf9-431c-82ab-88d0992cf8bb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.378356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f94ff-ecf9-431c-82ab-88d0992cf8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.378846 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f94ff-ecf9-431c-82ab-88d0992cf8bb None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.380194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f94ff-ecf9-431c-82ab-88d0992cf8bb None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:02.382520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f94ff-ecf9-431c-82ab-88d0992cf8bb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:02.385065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 188/375] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:02 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:02.391196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f94ff-ecf9-431c-82ab-88d0992cf8bb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:02.443842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 188/376] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:25:02.449169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb939fb7-2596-4661-a9f3-014a3cd189b8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:02.468513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:02.503079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f94ff-ecf9-431c-82ab-88d0992cf8bb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:02.545172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f5e3c-7693-4592-a911-946c36d48295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:02.546126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f5e3c-7693-4592-a911-946c36d48295 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.546601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f5e3c-7693-4592-a911-946c36d48295 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.547079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f5e3c-7693-4592-a911-946c36d48295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.548034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f5e3c-7693-4592-a911-946c36d48295 None admin] RBAC: Adding query filter params (group.id=71054dec6b5c40f185e534fbe0c143f8, role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:02.548541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f5e3c-7693-4592-a911-946c36d48295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:02.550211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f5e3c-7693-4592-a911-946c36d48295 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:02.557825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 189/377] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:02 2022] GET /identity/v3/role_assignments?group.id=71054dec6b5c40f185e534fbe0c143f8&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:02.605787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:02.616823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cd0a8-3be2-4ac1-8dba-b3e342d6c7ec None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.617161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cd0a8-3be2-4ac1-8dba-b3e342d6c7ec None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.617377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cd0a8-3be2-4ac1-8dba-b3e342d6c7ec None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.618888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 190/378] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:02.625718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22863c42-01ea-43a3-97a9-cc258b8cdd5a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.626371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22863c42-01ea-43a3-97a9-cc258b8cdd5a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.626866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22863c42-01ea-43a3-97a9-cc258b8cdd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.648522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-22863c42-01ea-43a3-97a9-cc258b8cdd5a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:02.650308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-22863c42-01ea-43a3-97a9-cc258b8cdd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:02.663674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb939fb7-2596-4661-a9f3-014a3cd189b8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:02.730536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ae237-2d0c-4ea8-b753-37de9b6ebf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:02.731308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ae237-2d0c-4ea8-b753-37de9b6ebf91 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.731592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ae237-2d0c-4ea8-b753-37de9b6ebf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.731862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ae237-2d0c-4ea8-b753-37de9b6ebf91 None admin] PATH_INFO: `/v3/projects/e7cfe98f92774b529ebade94e29001a3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.735312 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ae237-2d0c-4ea8-b753-37de9b6ebf91 None admin] RBAC: Authorizing `identity:get_project(project_id=e7cfe98f92774b529ebade94e29001a3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:02.736848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ae237-2d0c-4ea8-b753-37de9b6ebf91 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:02.738909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 189/379] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:02 2022] GET /identity/v3/projects/e7cfe98f92774b529ebade94e29001a3 => generated 344 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:02.740638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 191/380] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:02.745803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ae237-2d0c-4ea8-b753-37de9b6ebf91 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:02.770042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22863c42-01ea-43a3-97a9-cc258b8cdd5a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:02.851700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:02.885667 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ae237-2d0c-4ea8-b753-37de9b6ebf91 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:02.891574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22863c42-01ea-43a3-97a9-cc258b8cdd5a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:02.923373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ceafa-55ce-48be-bfb2-65402f7c01af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:02.924195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ceafa-55ce-48be-bfb2-65402f7c01af None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.924577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ceafa-55ce-48be-bfb2-65402f7c01af None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.924952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ceafa-55ce-48be-bfb2-65402f7c01af None admin] PATH_INFO: `/v3/groups/0abf4fe1ed8848dcaa24f9c9a99b3313` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.935730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ceafa-55ce-48be-bfb2-65402f7c01af None admin] RBAC: Authorizing `identity:get_group(group_id=0abf4fe1ed8848dcaa24f9c9a99b3313)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:02.937697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ceafa-55ce-48be-bfb2-65402f7c01af None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:02.938981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f61054-d7cb-4bac-b938-579039a80b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:02.939824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f61054-d7cb-4bac-b938-579039a80b3a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:02.940260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f61054-d7cb-4bac-b938-579039a80b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:02.940686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f61054-d7cb-4bac-b938-579039a80b3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:02.943341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 190/381] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:02 2022] GET /identity/v3/groups/0abf4fe1ed8848dcaa24f9c9a99b3313 => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:02.949306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ceafa-55ce-48be-bfb2-65402f7c01af None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:02.965653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f61054-d7cb-4bac-b938-579039a80b3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:02.974049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-a5f61054-d7cb-4bac-b938-579039a80b3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:02.977203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:02.978675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:25:02.979678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:02.982768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 192/382] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:02.988778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f61054-d7cb-4bac-b938-579039a80b3a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:03.151385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ceafa-55ce-48be-bfb2-65402f7c01af None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:03.155420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f61054-d7cb-4bac-b938-579039a80b3a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:03.194788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:03.195703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:03.196126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:03.196570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:03.197544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] RBAC: Adding query filter params (group.id=0abf4fe1ed8848dcaa24f9c9a99b3313, role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=e7cfe98f92774b529ebade94e29001a3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:03.198057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:03.202221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:03.209093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 191/383] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:02 2022] GET /identity/v3/role_assignments?group.id=0abf4fe1ed8848dcaa24f9c9a99b3313&scope.project.id=e7cfe98f92774b529ebade94e29001a3&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:03.222721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:03.230221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:03.231253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:03.231781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:03.232268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:03.233792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:03.234330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:03.237007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:03.253912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 193/384] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:25:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:03.260129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:03.370144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:03.416272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41970de-bd1d-40c8-8d86-bb3a1ab3fdef None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:03.464673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d522209-edd9-47a4-a833-cd7465a65976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:03.465343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d522209-edd9-47a4-a833-cd7465a65976 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:03.465621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d522209-edd9-47a4-a833-cd7465a65976 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:03.465889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d522209-edd9-47a4-a833-cd7465a65976 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:03.467014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d522209-edd9-47a4-a833-cd7465a65976 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:03.468508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d522209-edd9-47a4-a833-cd7465a65976 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:03.470389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 192/385] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:03 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:03.475967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:03.629023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad40e72-b356-4a84-b0bd-45964669abf9 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:03.676019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cad3e8-315a-4ebf-a275-7833fa55be64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:03.676876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cad3e8-315a-4ebf-a275-7833fa55be64 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:03.677379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cad3e8-315a-4ebf-a275-7833fa55be64 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:03.677798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cad3e8-315a-4ebf-a275-7833fa55be64 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:03.679618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cad3e8-315a-4ebf-a275-7833fa55be64 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:03.681270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cad3e8-315a-4ebf-a275-7833fa55be64 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:03.683483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 194/386] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:03 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:03.689151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d522209-edd9-47a4-a833-cd7465a65976 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:03.808253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d522209-edd9-47a4-a833-cd7465a65976 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:03.848362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae0ae5-e8f3-48fc-b674-ffa0a74dc0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:03.849034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0ae5-e8f3-48fc-b674-ffa0a74dc0e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:03.849341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0ae5-e8f3-48fc-b674-ffa0a74dc0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:03.849610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0ae5-e8f3-48fc-b674-ffa0a74dc0e5 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279/groups/71054dec6b5c40f185e534fbe0c143f8/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:03.851561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae0ae5-e8f3-48fc-b674-ffa0a74dc0e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d1b82d0c0604e3a98c466c903e60279, group_id=71054dec6b5c40f185e534fbe0c143f8, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:03.852982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae0ae5-e8f3-48fc-b674-ffa0a74dc0e5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:03.878090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 193/387] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 10 13:25:03 2022] PUT /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279/groups/71054dec6b5c40f185e534fbe0c143f8/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:04.279966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567a81-4585-4174-9d37-6429c5f9096f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:04.280828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567a81-4585-4174-9d37-6429c5f9096f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:04.281363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567a81-4585-4174-9d37-6429c5f9096f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:04.283317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 195/388] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:04.289333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7aa5-2a18-470b-ad93-0a74ccbe4519 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:04.289789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7aa5-2a18-470b-ad93-0a74ccbe4519 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:04.289880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7aa5-2a18-470b-ad93-0a74ccbe4519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:04.297202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:04.314914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2a6e7aa5-2a18-470b-ad93-0a74ccbe4519 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:04.316113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2a6e7aa5-2a18-470b-ad93-0a74ccbe4519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:04.407144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 194/389] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:04.442132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53567a81-4585-4174-9d37-6429c5f9096f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:04.529749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:04.599061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53567a81-4585-4174-9d37-6429c5f9096f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:04.651904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a4aee-0df1-4b30-8828-119e37774a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:04.652547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a4aee-0df1-4b30-8828-119e37774a7d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:04.652820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a4aee-0df1-4b30-8828-119e37774a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:04.653083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a4aee-0df1-4b30-8828-119e37774a7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:04.676390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a4aee-0df1-4b30-8828-119e37774a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2a6a4aee-0df1-4b30-8828-119e37774a7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:04.682190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:04.683113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:25:04.683913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:04.685009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 196/390] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:04.689549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e7aa5-2a18-470b-ad93-0a74ccbe4519 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:04.812027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e7aa5-2a18-470b-ad93-0a74ccbe4519 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:04.842187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:04.853692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dfacf-960c-444d-9d3c-0787e4071c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:04.854321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dfacf-960c-444d-9d3c-0787e4071c6d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:04.854594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dfacf-960c-444d-9d3c-0787e4071c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:04.854859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dfacf-960c-444d-9d3c-0787e4071c6d None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:04.855534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dfacf-960c-444d-9d3c-0787e4071c6d None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:04.855829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dfacf-960c-444d-9d3c-0787e4071c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:04.857206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dfacf-960c-444d-9d3c-0787e4071c6d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:04.871519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 195/391] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:25:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:04.872987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a4aee-0df1-4b30-8828-119e37774a7d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:05.010285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a4aee-0df1-4b30-8828-119e37774a7d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:05.039486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:05.068377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e2e18-3c18-46c4-9032-50acb6e6aea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:05.069510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e2e18-3c18-46c4-9032-50acb6e6aea6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.069784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e2e18-3c18-46c4-9032-50acb6e6aea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.070045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e2e18-3c18-46c4-9032-50acb6e6aea6 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.072465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191462d-779a-4211-8302-337364bdf198 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.072738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191462d-779a-4211-8302-337364bdf198 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.073007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191462d-779a-4211-8302-337364bdf198 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.074513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 196/392] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:05.074857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e2e18-3c18-46c4-9032-50acb6e6aea6 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:05.076437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e2e18-3c18-46c4-9032-50acb6e6aea6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:05.081670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 197/393] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:04 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:05.083029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed818-9733-47e1-91df-2f19e843cbdb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.083311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed818-9733-47e1-91df-2f19e843cbdb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.083578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed818-9733-47e1-91df-2f19e843cbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.091335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e2e18-3c18-46c4-9032-50acb6e6aea6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:05.116019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f92ed818-9733-47e1-91df-2f19e843cbdb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:05.117215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f92ed818-9733-47e1-91df-2f19e843cbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:05.217614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e2e18-3c18-46c4-9032-50acb6e6aea6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:05.261677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb50e9-dac9-4a97-9ed7-3266814f7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:05.262623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb50e9-dac9-4a97-9ed7-3266814f7fcd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.262908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb50e9-dac9-4a97-9ed7-3266814f7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.263176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb50e9-dac9-4a97-9ed7-3266814f7fcd None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.264306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb50e9-dac9-4a97-9ed7-3266814f7fcd None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:05.265775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb50e9-dac9-4a97-9ed7-3266814f7fcd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:05.278800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 198/394] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:05 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:05.280582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb50e9-dac9-4a97-9ed7-3266814f7fcd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:05.309881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 197/395] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:05.318946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc424-724b-4eba-a062-61ee56c1a1e5 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.319292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc424-724b-4eba-a062-61ee56c1a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.319752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc424-724b-4eba-a062-61ee56c1a1e5 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.322521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 198/396] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:05.333103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ca417-2972-4a17-86ff-bd819860e4bb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.333487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ca417-2972-4a17-86ff-bd819860e4bb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.333859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ca417-2972-4a17-86ff-bd819860e4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.351806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:05.365865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b00ca417-2972-4a17-86ff-bd819860e4bb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:05.367235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b00ca417-2972-4a17-86ff-bd819860e4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:05.400892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb50e9-dac9-4a97-9ed7-3266814f7fcd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:05.452243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2e0d0-2743-416a-9b4f-e06cc7250597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:05.453546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e0d0-2743-416a-9b4f-e06cc7250597 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.454125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e0d0-2743-416a-9b4f-e06cc7250597 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.455506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e0d0-2743-416a-9b4f-e06cc7250597 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/groups/71054dec6b5c40f185e534fbe0c143f8/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.460933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2e0d0-2743-416a-9b4f-e06cc7250597 None admin] RBAC: Authorizing `identity:create_grant(project_id=28871a3f7ee34b03bf66682bb0a217ec, group_id=71054dec6b5c40f185e534fbe0c143f8, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:05.463815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2e0d0-2743-416a-9b4f-e06cc7250597 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:05.473532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 199/397] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:05.481775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 199/398] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 10 13:25:05 2022] PUT /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/groups/71054dec6b5c40f185e534fbe0c143f8/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:25:05.485646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ca417-2972-4a17-86ff-bd819860e4bb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:05.518321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2e0d0-2743-416a-9b4f-e06cc7250597 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:05.589656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:05.671390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ca417-2972-4a17-86ff-bd819860e4bb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:05.711518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25222de1-31c7-4393-8725-08109aca2de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:05.712364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25222de1-31c7-4393-8725-08109aca2de5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.712822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25222de1-31c7-4393-8725-08109aca2de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.713190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25222de1-31c7-4393-8725-08109aca2de5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.725147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25222de1-31c7-4393-8725-08109aca2de5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-25222de1-31c7-4393-8725-08109aca2de5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:05.730010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:05.730908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:05.732216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:05.733452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2e0d0-2743-416a-9b4f-e06cc7250597 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:05.733540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 200/399] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:05.737531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25222de1-31c7-4393-8725-08109aca2de5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:05.745449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:05.790032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c85fd-b39b-4b01-8037-86402212554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:05.790849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c85fd-b39b-4b01-8037-86402212554c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.791229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c85fd-b39b-4b01-8037-86402212554c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.791590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c85fd-b39b-4b01-8037-86402212554c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.812203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c85fd-b39b-4b01-8037-86402212554c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-419c85fd-b39b-4b01-8037-86402212554c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:05.828796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:05.829819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:05.830606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:05.833352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 200/400] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:05.850472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c85fd-b39b-4b01-8037-86402212554c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:05.861747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25222de1-31c7-4393-8725-08109aca2de5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:05.878074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:05.901157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:05.902085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:05.902524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:05.902933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:05.903810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:05.904265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:05.905889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:05.914949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 201/401] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:05.920650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:06.028551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf77c7-f620-44fa-8ca3-4e9f2fffdaa0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:06.065453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adc047-f0f6-4b24-9b3f-41727f7e2ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:06.066417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adc047-f0f6-4b24-9b3f-41727f7e2ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.066855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adc047-f0f6-4b24-9b3f-41727f7e2ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.067271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adc047-f0f6-4b24-9b3f-41727f7e2ee7 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.068616 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2adc047-f0f6-4b24-9b3f-41727f7e2ee7 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:06.070734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2adc047-f0f6-4b24-9b3f-41727f7e2ee7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:06.072596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 202/402] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:05 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:06.073922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a668f-919d-4542-b065-027b1975a02a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.074376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a668f-919d-4542-b065-027b1975a02a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.074648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a668f-919d-4542-b065-027b1975a02a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.078332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c85fd-b39b-4b01-8037-86402212554c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:06.081281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 203/403] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:06.081535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a668f-919d-4542-b065-027b1975a02a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:06.134888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959450ab-7d25-42fb-920e-7e4ec9c077b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:06.136143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959450ab-7d25-42fb-920e-7e4ec9c077b7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.136428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959450ab-7d25-42fb-920e-7e4ec9c077b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.136701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959450ab-7d25-42fb-920e-7e4ec9c077b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.137890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959450ab-7d25-42fb-920e-7e4ec9c077b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:06.138197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959450ab-7d25-42fb-920e-7e4ec9c077b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:06.140351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959450ab-7d25-42fb-920e-7e4ec9c077b7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:06.141960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:06.159967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 201/404] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:06.161254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102e64-7498-41fb-a4e0-3fe3b68e20e3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.161532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102e64-7498-41fb-a4e0-3fe3b68e20e3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.161800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102e64-7498-41fb-a4e0-3fe3b68e20e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.191015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-31102e64-7498-41fb-a4e0-3fe3b68e20e3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:06.192753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-31102e64-7498-41fb-a4e0-3fe3b68e20e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:06.227810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a668f-919d-4542-b065-027b1975a02a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:06.232909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:06.285078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffa43f-4a5a-46e4-97e8-3c71a2cbc594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:06.285780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffa43f-4a5a-46e4-97e8-3c71a2cbc594 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.286059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffa43f-4a5a-46e4-97e8-3c71a2cbc594 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.286328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffa43f-4a5a-46e4-97e8-3c71a2cbc594 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.287534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffa43f-4a5a-46e4-97e8-3c71a2cbc594 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:06.289125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffa43f-4a5a-46e4-97e8-3c71a2cbc594 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:06.290177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 202/405] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:06.291648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31102e64-7498-41fb-a4e0-3fe3b68e20e3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:06.295961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 204/406] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:06 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:06.296907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffa43f-4a5a-46e4-97e8-3c71a2cbc594 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:06.413311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffa43f-4a5a-46e4-97e8-3c71a2cbc594 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:06.429215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:06.436836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31102e64-7498-41fb-a4e0-3fe3b68e20e3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:06.455471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2af73-c6b0-4a6d-adca-57495a0fc0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:06.456144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2af73-c6b0-4a6d-adca-57495a0fc0e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.456421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2af73-c6b0-4a6d-adca-57495a0fc0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.456695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2af73-c6b0-4a6d-adca-57495a0fc0e3 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279/groups/71054dec6b5c40f185e534fbe0c143f8/roles/d0ccb065ab134c66b5033b67d0bb86b6` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.458961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2af73-c6b0-4a6d-adca-57495a0fc0e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d1b82d0c0604e3a98c466c903e60279, group_id=71054dec6b5c40f185e534fbe0c143f8, role_id=d0ccb065ab134c66b5033b67d0bb86b6)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:06.461052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2af73-c6b0-4a6d-adca-57495a0fc0e3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:06.481645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5556c-1dae-4946-9c86-ceb0d89a62bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:06.482275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5556c-1dae-4946-9c86-ceb0d89a62bf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.482860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5556c-1dae-4946-9c86-ceb0d89a62bf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.483087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5556c-1dae-4946-9c86-ceb0d89a62bf None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.483563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 205/407] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 10 13:25:06 2022] PUT /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279/groups/71054dec6b5c40f185e534fbe0c143f8/roles/d0ccb065ab134c66b5033b67d0bb86b6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:06.484926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2af73-c6b0-4a6d-adca-57495a0fc0e3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:06.486056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5556c-1dae-4946-9c86-ceb0d89a62bf None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:06.487575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5556c-1dae-4946-9c86-ceb0d89a62bf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:06.489285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 203/408] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:06 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:06.491072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44e729-a9ac-404e-8874-f17509f63a4c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.491359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44e729-a9ac-404e-8874-f17509f63a4c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.491632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44e729-a9ac-404e-8874-f17509f63a4c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.493199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 204/409] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:06.498966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9309158-cd72-4903-85a9-a028633d96a1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.499423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9309158-cd72-4903-85a9-a028633d96a1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.499691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9309158-cd72-4903-85a9-a028633d96a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.518860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b9309158-cd72-4903-85a9-a028633d96a1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:06.520107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b9309158-cd72-4903-85a9-a028633d96a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:06.550070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:06.622039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2af73-c6b0-4a6d-adca-57495a0fc0e3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:06.637650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 205/410] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:06.638659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9309158-cd72-4903-85a9-a028633d96a1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:06.688888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d0491-9186-4eee-861c-5993f921e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:06.689553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d0491-9186-4eee-861c-5993f921e6ed None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.689832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d0491-9186-4eee-861c-5993f921e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.690099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d0491-9186-4eee-861c-5993f921e6ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.702598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d0491-9186-4eee-861c-5993f921e6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-291d0491-9186-4eee-861c-5993f921e6ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:06.707811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:06.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:06.710107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 206/411] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:06.710771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d0491-9186-4eee-861c-5993f921e6ed None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:06.760838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:06.825576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9309158-cd72-4903-85a9-a028633d96a1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:06.892688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05203ffb-568e-4bf7-a30e-79bf0f56f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:06.893378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05203ffb-568e-4bf7-a30e-79bf0f56f873 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.893656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05203ffb-568e-4bf7-a30e-79bf0f56f873 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.893925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05203ffb-568e-4bf7-a30e-79bf0f56f873 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.895211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05203ffb-568e-4bf7-a30e-79bf0f56f873 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:06.897870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d0491-9186-4eee-861c-5993f921e6ed None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:06.900874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05203ffb-568e-4bf7-a30e-79bf0f56f873 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:06.909389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 206/412] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:06 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:06.910333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05203ffb-568e-4bf7-a30e-79bf0f56f873 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:06.965692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:06.970499 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282622d-65eb-41a3-85e1-a9397b5f5ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:06.971436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282622d-65eb-41a3-85e1-a9397b5f5ffd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:06.972324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282622d-65eb-41a3-85e1-a9397b5f5ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:06.973788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282622d-65eb-41a3-85e1-a9397b5f5ffd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:06.991410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282622d-65eb-41a3-85e1-a9397b5f5ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d282622d-65eb-41a3-85e1-a9397b5f5ffd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:06.998813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:06.999753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:07.000558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 207/413] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:07.001202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282622d-65eb-41a3-85e1-a9397b5f5ffd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:07.086406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05203ffb-568e-4bf7-a30e-79bf0f56f873 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:07.158915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:07.159532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.161212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.162235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:07.162954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:07.163255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:07.164649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:07.185963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:07.188695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 207/414] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:07.195221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:07.201975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282622d-65eb-41a3-85e1-a9397b5f5ffd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:07.252945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f16e-5f4c-46dd-ab43-91be8e3e47e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:07.254170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f16e-5f4c-46dd-ab43-91be8e3e47e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.254665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f16e-5f4c-46dd-ab43-91be8e3e47e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.255986 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f16e-5f4c-46dd-ab43-91be8e3e47e0 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/groups/71054dec6b5c40f185e534fbe0c143f8/roles/d0ccb065ab134c66b5033b67d0bb86b6` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:07.259154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81f16e-5f4c-46dd-ab43-91be8e3e47e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=28871a3f7ee34b03bf66682bb0a217ec, group_id=71054dec6b5c40f185e534fbe0c143f8, role_id=d0ccb065ab134c66b5033b67d0bb86b6)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:07.261417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81f16e-5f4c-46dd-ab43-91be8e3e47e0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:07.283638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 208/415] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 10 13:25:06 2022] PUT /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec/groups/71054dec6b5c40f185e534fbe0c143f8/roles/d0ccb065ab134c66b5033b67d0bb86b6 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:07.284831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f16e-5f4c-46dd-ab43-91be8e3e47e0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:07.370791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:07.391792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab301e-5b26-4b33-a012-48e9c397ffd8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:07.430012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f16e-5f4c-46dd-ab43-91be8e3e47e0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:07.487893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86999ece-8259-4b8c-8d14-d27c5004ff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:07.488909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86999ece-8259-4b8c-8d14-d27c5004ff3b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.489689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86999ece-8259-4b8c-8d14-d27c5004ff3b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.490247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86999ece-8259-4b8c-8d14-d27c5004ff3b None admin] PATH_INFO: `/v3/groups/0abf4fe1ed8848dcaa24f9c9a99b3313` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:07.491279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:07.491984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.492309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.492530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:07.493153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:07.493480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:07.494159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86999ece-8259-4b8c-8d14-d27c5004ff3b None admin] RBAC: Authorizing `identity:get_group(group_id=0abf4fe1ed8848dcaa24f9c9a99b3313)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:07.494877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:07.496047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86999ece-8259-4b8c-8d14-d27c5004ff3b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:07.498547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 209/416] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:07 2022] GET /identity/v3/groups/0abf4fe1ed8848dcaa24f9c9a99b3313 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:07.505929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86999ece-8259-4b8c-8d14-d27c5004ff3b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:07.516718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 208/417] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:25:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:07.527471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:07.571401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:07.670096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfee09e-d1da-4f08-b599-333b28144d96 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:07.684389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86999ece-8259-4b8c-8d14-d27c5004ff3b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:07.722520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892e490-8bcb-4495-9c2a-3e0612f91098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:07.723196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892e490-8bcb-4495-9c2a-3e0612f91098 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.723476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892e490-8bcb-4495-9c2a-3e0612f91098 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.723748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892e490-8bcb-4495-9c2a-3e0612f91098 None admin] PATH_INFO: `/v3/projects/e7cfe98f92774b529ebade94e29001a3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:07.724942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892e490-8bcb-4495-9c2a-3e0612f91098 None admin] RBAC: Authorizing `identity:get_project(project_id=e7cfe98f92774b529ebade94e29001a3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:07.726433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892e490-8bcb-4495-9c2a-3e0612f91098 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:07.728396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 210/418] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:07 2022] GET /identity/v3/projects/e7cfe98f92774b529ebade94e29001a3 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:07.734236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892e490-8bcb-4495-9c2a-3e0612f91098 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:07.754359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585f703-7266-43fb-9e52-79b2dad8449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:07.754981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585f703-7266-43fb-9e52-79b2dad8449c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.755275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585f703-7266-43fb-9e52-79b2dad8449c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.755619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585f703-7266-43fb-9e52-79b2dad8449c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:07.810950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:07.891952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585f703-7266-43fb-9e52-79b2dad8449c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:07.894439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585f703-7266-43fb-9e52-79b2dad8449c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:07.912562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:07.927201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892e490-8bcb-4495-9c2a-3e0612f91098 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:07.964046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb831f-7d89-4b5e-af4d-b94be2a568b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:07.964702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb831f-7d89-4b5e-af4d-b94be2a568b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.964973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb831f-7d89-4b5e-af4d-b94be2a568b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.965258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb831f-7d89-4b5e-af4d-b94be2a568b5 None admin] PATH_INFO: `/v3/projects/e7cfe98f92774b529ebade94e29001a3/groups/0abf4fe1ed8848dcaa24f9c9a99b3313/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:07.967279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb831f-7d89-4b5e-af4d-b94be2a568b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7cfe98f92774b529ebade94e29001a3, group_id=0abf4fe1ed8848dcaa24f9c9a99b3313, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:07.968791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb831f-7d89-4b5e-af4d-b94be2a568b5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:07.984335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 211/419] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 10 13:25:07 2022] PUT /identity/v3/projects/e7cfe98f92774b529ebade94e29001a3/groups/0abf4fe1ed8848dcaa24f9c9a99b3313/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:25:07.985969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b79c7-329e-420f-8482-11711251e148 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.986299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b79c7-329e-420f-8482-11711251e148 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.986610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b79c7-329e-420f-8482-11711251e148 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:07.993360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 212/420] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:07.995027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baf4dc-14e3-4c4c-bf2c-44fee7751cff None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:07.995475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baf4dc-14e3-4c4c-bf2c-44fee7751cff None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:07.995746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baf4dc-14e3-4c4c-bf2c-44fee7751cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.014258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 209/421] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:08.020021 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585f703-7266-43fb-9e52-79b2dad8449c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:08.020204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e4baf4dc-14e3-4c4c-bf2c-44fee7751cff None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:08.021637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e4baf4dc-14e3-4c4c-bf2c-44fee7751cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:08.102033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 213/422] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:08.118971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:08.128232 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baf4dc-14e3-4c4c-bf2c-44fee7751cff None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:08.209927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585f703-7266-43fb-9e52-79b2dad8449c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:08.225628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:08.254355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baf4dc-14e3-4c4c-bf2c-44fee7751cff None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:08.282410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedf88a-349e-4ae5-bf6e-0e7f1e0324c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:08.283155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedf88a-349e-4ae5-bf6e-0e7f1e0324c1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:08.283532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedf88a-349e-4ae5-bf6e-0e7f1e0324c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:08.283810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedf88a-349e-4ae5-bf6e-0e7f1e0324c1 None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.284935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedf88a-349e-4ae5-bf6e-0e7f1e0324c1 None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:08.286611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedf88a-349e-4ae5-bf6e-0e7f1e0324c1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:08.289089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 210/423] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:08 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:08.292547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4c755-7def-48b1-b85b-21237fcc3958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:08.293181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c755-7def-48b1-b85b-21237fcc3958 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:08.293483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c755-7def-48b1-b85b-21237fcc3958 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:08.293749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c755-7def-48b1-b85b-21237fcc3958 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.295758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedf88a-349e-4ae5-bf6e-0e7f1e0324c1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:08.305889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4c755-7def-48b1-b85b-21237fcc3958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-34c4c755-7def-48b1-b85b-21237fcc3958 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:08.310811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:08.312876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:08.314983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 214/424] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:08.320993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4c755-7def-48b1-b85b-21237fcc3958 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:08.408023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedf88a-349e-4ae5-bf6e-0e7f1e0324c1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:08.425914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4c755-7def-48b1-b85b-21237fcc3958 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:08.445612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:08.446960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e5ca0-1826-4cd6-9e81-cceb10e6aaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:08.447711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e5ca0-1826-4cd6-9e81-cceb10e6aaae None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:08.448085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e5ca0-1826-4cd6-9e81-cceb10e6aaae None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:08.449579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e5ca0-1826-4cd6-9e81-cceb10e6aaae None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.449579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e5ca0-1826-4cd6-9e81-cceb10e6aaae None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:08.451484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e5ca0-1826-4cd6-9e81-cceb10e6aaae None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:08.453569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 211/425] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:08 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:08.459600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e5ca0-1826-4cd6-9e81-cceb10e6aaae None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:08.470436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:08.471077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:08.475320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:08.475611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.476347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:08.476651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:08.478054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:08.488957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:08.502355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 215/426] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:25:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:08.512476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:08.666560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0312-b92a-4cd8-9cdb-905c08711100 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:08.718882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa613b0-b20c-467a-9a73-4d5fdce3ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:08.719527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa613b0-b20c-467a-9a73-4d5fdce3ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:08.719806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa613b0-b20c-467a-9a73-4d5fdce3ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:08.720078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa613b0-b20c-467a-9a73-4d5fdce3ac7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.782169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e5ca0-1826-4cd6-9e81-cceb10e6aaae None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:08.810997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:08.814903 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa613b0-b20c-467a-9a73-4d5fdce3ac7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:08.817268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa613b0-b20c-467a-9a73-4d5fdce3ac7c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:08.840950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875ac19-a493-4b21-935a-3b0b992b1976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:08.841587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875ac19-a493-4b21-935a-3b0b992b1976 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:08.841871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875ac19-a493-4b21-935a-3b0b992b1976 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:08.842148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875ac19-a493-4b21-935a-3b0b992b1976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.842888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875ac19-a493-4b21-935a-3b0b992b1976 None admin] RBAC: Adding query filter params (group.id=71054dec6b5c40f185e534fbe0c143f8, role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=8d1b82d0c0604e3a98c466c903e60279) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:08.843197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875ac19-a493-4b21-935a-3b0b992b1976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:08.844710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875ac19-a493-4b21-935a-3b0b992b1976 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:08.852016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 212/427] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:08 2022] GET /identity/v3/role_assignments?group.id=71054dec6b5c40f185e534fbe0c143f8&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 611 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:08.853584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e214470-6ae5-4b56-817d-9e7ef911d098 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:08.853903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e214470-6ae5-4b56-817d-9e7ef911d098 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:08.854194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e214470-6ae5-4b56-817d-9e7ef911d098 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.860470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 213/428] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:08.862515 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae837-8b3f-487f-88c0-517547f5ab3e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:08.862898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae837-8b3f-487f-88c0-517547f5ab3e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:08.863046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae837-8b3f-487f-88c0-517547f5ab3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:08.882451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-307ae837-8b3f-487f-88c0-517547f5ab3e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:08.883818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-307ae837-8b3f-487f-88c0-517547f5ab3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:08.950801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 216/429] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:25:08.957721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa613b0-b20c-467a-9a73-4d5fdce3ac7c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:09.003321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:09.063434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 214/430] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:09.090514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ae837-8b3f-487f-88c0-517547f5ab3e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:09.094864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:09.098448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa613b0-b20c-467a-9a73-4d5fdce3ac7c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:09.143587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292a360-4ea2-4c53-8ccb-f6da2c71206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:09.144578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a360-4ea2-4c53-8ccb-f6da2c71206d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:09.145092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a360-4ea2-4c53-8ccb-f6da2c71206d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:09.145663 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a360-4ea2-4c53-8ccb-f6da2c71206d None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:09.147325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292a360-4ea2-4c53-8ccb-f6da2c71206d None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:09.148967 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292a360-4ea2-4c53-8ccb-f6da2c71206d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:09.151418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 217/431] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:08 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:09.160034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292a360-4ea2-4c53-8ccb-f6da2c71206d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:09.210301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ae837-8b3f-487f-88c0-517547f5ab3e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:09.264934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3aeb1-0461-4ef4-94a5-5f6c7ec46b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:09.265611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3aeb1-0461-4ef4-94a5-5f6c7ec46b27 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:09.265891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3aeb1-0461-4ef4-94a5-5f6c7ec46b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:09.266164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3aeb1-0461-4ef4-94a5-5f6c7ec46b27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:09.284300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3aeb1-0461-4ef4-94a5-5f6c7ec46b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-add3aeb1-0461-4ef4-94a5-5f6c7ec46b27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:09.290059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:09.291076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:09.291858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:09.294496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 215/432] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:09.300073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292a360-4ea2-4c53-8ccb-f6da2c71206d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:09.300476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3aeb1-0461-4ef4-94a5-5f6c7ec46b27 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:09.356890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1adc8-694d-4629-833e-0b0b1d437782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:09.357926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1adc8-694d-4629-833e-0b0b1d437782 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:09.358515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1adc8-694d-4629-833e-0b0b1d437782 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:09.358853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1adc8-694d-4629-833e-0b0b1d437782 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:09.361126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1adc8-694d-4629-833e-0b0b1d437782 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:09.362758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1adc8-694d-4629-833e-0b0b1d437782 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:09.364822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 218/433] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:09 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:09.371165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1adc8-694d-4629-833e-0b0b1d437782 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:09.377294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:09.418382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:09.456821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3aeb1-0461-4ef4-94a5-5f6c7ec46b27 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:09.498657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:09.499393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1adc8-694d-4629-833e-0b0b1d437782 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:09.500182 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:09.500670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:09.501129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:09.502060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:09.502576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:09.504169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:09.516974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 216/434] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:09.524849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:09.538373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:09.538995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:09.539269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:09.539535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:09.540270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] RBAC: Adding query filter params (group.id=71054dec6b5c40f185e534fbe0c143f8, role.id=49b24f3e5ebc4b08b538432eda6b4b5d, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:09.540571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:09.542921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:09.549430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 219/435] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:09 2022] GET /identity/v3/role_assignments?group.id=71054dec6b5c40f185e534fbe0c143f8&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=49b24f3e5ebc4b08b538432eda6b4b5d => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:09.724238 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ec46f-2ab4-4491-8cce-c23c637af6db None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:09.733258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:09.766355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a13e-a863-4cfd-a36f-5312f1f11358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:09.766885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a13e-a863-4cfd-a36f-5312f1f11358 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:09.767159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a13e-a863-4cfd-a36f-5312f1f11358 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:09.767427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a13e-a863-4cfd-a36f-5312f1f11358 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:09.839647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84a13e-a863-4cfd-a36f-5312f1f11358 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:09.841128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84a13e-a863-4cfd-a36f-5312f1f11358 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:09.948657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 217/436] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:25:09.956644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.030639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa204713-d887-44ea-8b25-985e107da856 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.030876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa204713-d887-44ea-8b25-985e107da856 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.031270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa204713-d887-44ea-8b25-985e107da856 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.032823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 218/437] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:10.038628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f14e-ba6d-4dd8-bda9-d8bcada36489 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.038975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f14e-ba6d-4dd8-bda9-d8bcada36489 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.039331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f14e-ba6d-4dd8-bda9-d8bcada36489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.056033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.070702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8638f14e-ba6d-4dd8-bda9-d8bcada36489 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.073726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8638f14e-ba6d-4dd8-bda9-d8bcada36489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.103684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.110756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d148-6fb5-452b-850f-bdb8165b3b1d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.156801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee3a5a-9bf4-4fbd-98c8-70ca26d6c56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.157724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee3a5a-9bf4-4fbd-98c8-70ca26d6c56d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.157831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 219/438] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:10.157909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee3a5a-9bf4-4fbd-98c8-70ca26d6c56d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.158092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee3a5a-9bf4-4fbd-98c8-70ca26d6c56d None admin] PATH_INFO: `/v3/projects/8d1b82d0c0604e3a98c466c903e60279` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.159264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee3a5a-9bf4-4fbd-98c8-70ca26d6c56d None admin] RBAC: Authorizing `identity:get_project(project_id=8d1b82d0c0604e3a98c466c903e60279)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.160732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee3a5a-9bf4-4fbd-98c8-70ca26d6c56d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.167263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 220/439] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:09 2022] GET /identity/v3/projects/8d1b82d0c0604e3a98c466c903e60279 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:10.168455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638f14e-ba6d-4dd8-bda9-d8bcada36489 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.182910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee3a5a-9bf4-4fbd-98c8-70ca26d6c56d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.306346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638f14e-ba6d-4dd8-bda9-d8bcada36489 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.316183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee3a5a-9bf4-4fbd-98c8-70ca26d6c56d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.358919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfebe0f-6a93-4bf8-b117-7d809c15b1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.359721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfebe0f-6a93-4bf8-b117-7d809c15b1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.359830 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfebe0f-6a93-4bf8-b117-7d809c15b1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.360090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfebe0f-6a93-4bf8-b117-7d809c15b1e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.366071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d8cf8-9523-449e-94cb-ab143a053852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.366989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d8cf8-9523-449e-94cb-ab143a053852 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.368208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d8cf8-9523-449e-94cb-ab143a053852 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.369310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d8cf8-9523-449e-94cb-ab143a053852 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.370637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d8cf8-9523-449e-94cb-ab143a053852 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.371854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfebe0f-6a93-4bf8-b117-7d809c15b1e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:10.372566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d8cf8-9523-449e-94cb-ab143a053852 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.375753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 220/440] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-9dfebe0f-6a93-4bf8-b117-7d809c15b1e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:10.377290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:10.378154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:10.378933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 221/441] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:10.382247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d8cf8-9523-449e-94cb-ab143a053852 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.386162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.388091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfebe0f-6a93-4bf8-b117-7d809c15b1e9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.421609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.493083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d8cf8-9523-449e-94cb-ab143a053852 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.501755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfebe0f-6a93-4bf8-b117-7d809c15b1e9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.529566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.530173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.530448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.530711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.531490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] RBAC: Adding query filter params (group.id=71054dec6b5c40f185e534fbe0c143f8, role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=8d1b82d0c0604e3a98c466c903e60279) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:10.531733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.533174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.539917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 221/442] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/role_assignments?group.id=71054dec6b5c40f185e534fbe0c143f8&scope.project.id=8d1b82d0c0604e3a98c466c903e60279&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:10.541680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07002aad-9564-4940-82d8-81495dec9498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.542300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07002aad-9564-4940-82d8-81495dec9498 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.542576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07002aad-9564-4940-82d8-81495dec9498 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.542845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07002aad-9564-4940-82d8-81495dec9498 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.543529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07002aad-9564-4940-82d8-81495dec9498 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:10.543829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07002aad-9564-4940-82d8-81495dec9498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.545338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07002aad-9564-4940-82d8-81495dec9498 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.555575 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 222/443] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:10.561355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.625774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b6512-b487-4ffd-9204-ba7a8fde22b4 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.626982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b6512-b487-4ffd-9204-ba7a8fde22b4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.627486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b6512-b487-4ffd-9204-ba7a8fde22b4 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.630701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 223/444] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:10.636782 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883c57-c708-4608-93cf-b0612ea207bc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.637072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883c57-c708-4608-93cf-b0612ea207bc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.637357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883c57-c708-4608-93cf-b0612ea207bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.649601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.663132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b9883c57-c708-4608-93cf-b0612ea207bc None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:10.665113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b9883c57-c708-4608-93cf-b0612ea207bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:10.694711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bec344-4c5e-48ce-a0e1-d1a59ae22b89 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.735082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574cf33-f2bd-4fd0-b652-dea4bfde25f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.735946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574cf33-f2bd-4fd0-b652-dea4bfde25f0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.736222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574cf33-f2bd-4fd0-b652-dea4bfde25f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.736495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574cf33-f2bd-4fd0-b652-dea4bfde25f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.758897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 224/445] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:10.782213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9883c57-c708-4608-93cf-b0612ea207bc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.809956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.820879 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574cf33-f2bd-4fd0-b652-dea4bfde25f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:10.822337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574cf33-f2bd-4fd0-b652-dea4bfde25f0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:10.914854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:10.916920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9883c57-c708-4608-93cf-b0612ea207bc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:10.954991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff259e-bcf0-4a89-95bf-c6b7a37c5942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:10.955483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 222/446] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:10.955961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff259e-bcf0-4a89-95bf-c6b7a37c5942 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:10.956238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff259e-bcf0-4a89-95bf-c6b7a37c5942 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:10.956606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff259e-bcf0-4a89-95bf-c6b7a37c5942 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:10.961495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574cf33-f2bd-4fd0-b652-dea4bfde25f0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:10.970503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff259e-bcf0-4a89-95bf-c6b7a37c5942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7bff259e-bcf0-4a89-95bf-c6b7a37c5942 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:10.975306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:10.976166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:10.976958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 225/447] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:10.982319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff259e-bcf0-4a89-95bf-c6b7a37c5942 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.081479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.087079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574cf33-f2bd-4fd0-b652-dea4bfde25f0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.100868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff259e-bcf0-4a89-95bf-c6b7a37c5942 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.123066 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d7f64-138f-40d3-a0b2-f12d6732d3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.123689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d7f64-138f-40d3-a0b2-f12d6732d3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.123961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d7f64-138f-40d3-a0b2-f12d6732d3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.124235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d7f64-138f-40d3-a0b2-f12d6732d3e8 None admin] PATH_INFO: `/v3/projects/28871a3f7ee34b03bf66682bb0a217ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.125366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d7f64-138f-40d3-a0b2-f12d6732d3e8 None admin] RBAC: Authorizing `identity:get_project(project_id=28871a3f7ee34b03bf66682bb0a217ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.126813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d7f64-138f-40d3-a0b2-f12d6732d3e8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.128580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 223/448] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/projects/28871a3f7ee34b03bf66682bb0a217ec => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:11.133832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d7f64-138f-40d3-a0b2-f12d6732d3e8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.134381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.134979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.135249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.135510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.136168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:11.136462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.137848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.148888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 226/449] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:11.154455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.168049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.235233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d7f64-138f-40d3-a0b2-f12d6732d3e8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.258560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ec0b6-e6bc-4868-9622-de5566138e33 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.268753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0936fe-110d-4993-a2bd-9d3cb2ea8f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.269405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0936fe-110d-4993-a2bd-9d3cb2ea8f69 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.269685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0936fe-110d-4993-a2bd-9d3cb2ea8f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.269951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0936fe-110d-4993-a2bd-9d3cb2ea8f69 None admin] PATH_INFO: `/v3/groups/71054dec6b5c40f185e534fbe0c143f8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.271047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0936fe-110d-4993-a2bd-9d3cb2ea8f69 None admin] RBAC: Authorizing `identity:get_group(group_id=71054dec6b5c40f185e534fbe0c143f8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.272518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0936fe-110d-4993-a2bd-9d3cb2ea8f69 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.274432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 224/450] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/groups/71054dec6b5c40f185e534fbe0c143f8 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:11.279922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0936fe-110d-4993-a2bd-9d3cb2ea8f69 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.293712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2584d0-0089-41a9-a23e-48ee5213c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.294318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2584d0-0089-41a9-a23e-48ee5213c455 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.294589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2584d0-0089-41a9-a23e-48ee5213c455 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.294851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2584d0-0089-41a9-a23e-48ee5213c455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.307010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.361661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2584d0-0089-41a9-a23e-48ee5213c455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.363029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2584d0-0089-41a9-a23e-48ee5213c455 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.379617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0936fe-110d-4993-a2bd-9d3cb2ea8f69 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.381964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.413208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.414531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.414809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.415742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.416472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] RBAC: Adding query filter params (group.id=71054dec6b5c40f185e534fbe0c143f8, role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=28871a3f7ee34b03bf66682bb0a217ec) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:11.416773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.418304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.424018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 225/451] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/role_assignments?group.id=71054dec6b5c40f185e534fbe0c143f8&scope.project.id=28871a3f7ee34b03bf66682bb0a217ec&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:11.463064 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 227/452] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:25:11.468790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.570538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6f2c0-814a-4ac0-af07-73106a3b5311 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.604281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6b0d3-d07e-491a-8173-8f2b8ded3894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.604899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6b0d3-d07e-491a-8173-8f2b8ded3894 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.605169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6b0d3-d07e-491a-8173-8f2b8ded3894 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.605461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6b0d3-d07e-491a-8173-8f2b8ded3894 None admin] PATH_INFO: `/v3/projects/e7cfe98f92774b529ebade94e29001a3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.606561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6b0d3-d07e-491a-8173-8f2b8ded3894 None admin] RBAC: Authorizing `identity:get_project(project_id=e7cfe98f92774b529ebade94e29001a3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.608254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6b0d3-d07e-491a-8173-8f2b8ded3894 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.609873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 226/453] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/projects/e7cfe98f92774b529ebade94e29001a3 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:11.615682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2584d0-0089-41a9-a23e-48ee5213c455 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.661015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.719740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2584d0-0089-41a9-a23e-48ee5213c455 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.755740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4d173-9504-44dd-8475-d451084f1cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.757157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4d173-9504-44dd-8475-d451084f1cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.757601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4d173-9504-44dd-8475-d451084f1cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.757992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4d173-9504-44dd-8475-d451084f1cd5 None admin] PATH_INFO: `/v3/groups/0abf4fe1ed8848dcaa24f9c9a99b3313` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.759319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4d173-9504-44dd-8475-d451084f1cd5 None admin] RBAC: Authorizing `identity:get_group(group_id=0abf4fe1ed8848dcaa24f9c9a99b3313)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.760747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4d173-9504-44dd-8475-d451084f1cd5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.762775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 228/454] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/groups/0abf4fe1ed8848dcaa24f9c9a99b3313 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:11.769036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6b0d3-d07e-491a-8173-8f2b8ded3894 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:11.803693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:11.865350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6b0d3-d07e-491a-8173-8f2b8ded3894 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:11.898236 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b3d20-a547-4218-a07c-ebdccf5a76f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:11.898831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3d20-a547-4218-a07c-ebdccf5a76f5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:11.899103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3d20-a547-4218-a07c-ebdccf5a76f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:11.899368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3d20-a547-4218-a07c-ebdccf5a76f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:11.900093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b3d20-a547-4218-a07c-ebdccf5a76f5 None admin] RBAC: Adding query filter params (group.id=0abf4fe1ed8848dcaa24f9c9a99b3313, role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=e7cfe98f92774b529ebade94e29001a3) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:11.900388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b3d20-a547-4218-a07c-ebdccf5a76f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:11.901844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b3d20-a547-4218-a07c-ebdccf5a76f5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:11.908062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 227/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 10 13:25:11 2022] GET /identity/v3/role_assignments?group.id=0abf4fe1ed8848dcaa24f9c9a99b3313&scope.project.id=e7cfe98f92774b529ebade94e29001a3&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:14.452880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fa48b-c968-4c94-a4d0-c117324d9173 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.453421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fa48b-c968-4c94-a4d0-c117324d9173 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.453496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fa48b-c968-4c94-a4d0-c117324d9173 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.455047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 229/456] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:14.461436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5b4c-f9d2-4355-99be-b029da354476 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.461922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5b4c-f9d2-4355-99be-b029da354476 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.461998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5b4c-f9d2-4355-99be-b029da354476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.494888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-633c5b4c-f9d2-4355-99be-b029da354476 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:14.496159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-633c5b4c-f9d2-4355-99be-b029da354476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:14.572661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e025e-a0d6-4581-b734-fb14575ce101 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.572846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e025e-a0d6-4581-b734-fb14575ce101 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.573118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e025e-a0d6-4581-b734-fb14575ce101 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.575806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 230/457] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:14.580119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea8fb-d7bf-4bd8-ac5b-f91a1a58319c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.580403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea8fb-d7bf-4bd8-ac5b-f91a1a58319c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.580675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea8fb-d7bf-4bd8-ac5b-f91a1a58319c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.603298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d7fea8fb-d7bf-4bd8-ac5b-f91a1a58319c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:14.604840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d7fea8fb-d7bf-4bd8-ac5b-f91a1a58319c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:14.608910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 228/458] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:14.633918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c5b4c-f9d2-4355-99be-b029da354476 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.692126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 231/459] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:14.700088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:14.717457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fea8fb-d7bf-4bd8-ac5b-f91a1a58319c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.764459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:14.799518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c5b4c-f9d2-4355-99be-b029da354476 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.838268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77c785-6619-4115-bdb4-c333a77979ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.839948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77c785-6619-4115-bdb4-c333a77979ec None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.840226 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77c785-6619-4115-bdb4-c333a77979ec None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.840495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77c785-6619-4115-bdb4-c333a77979ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.846422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fea8fb-d7bf-4bd8-ac5b-f91a1a58319c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:14.858610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77c785-6619-4115-bdb4-c333a77979ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.861062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77c785-6619-4115-bdb4-c333a77979ec None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5d77c785-6619-4115-bdb4-c333a77979ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:14.879572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:14.880559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:14.881551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:14.881551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:14.881551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:14.886149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 229/460] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:14.887779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cbdc3-fb3d-41c3-beab-8c9f3e30f436 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.888063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cbdc3-fb3d-41c3-beab-8c9f3e30f436 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.888338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cbdc3-fb3d-41c3-beab-8c9f3e30f436 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.893320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 230/461] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:14.894036 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aff6ad-25f2-49a7-84ef-2b14c31d1ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:14.894191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cbdc3-fb3d-41c3-beab-8c9f3e30f436 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:14.895061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aff6ad-25f2-49a7-84ef-2b14c31d1ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.895538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aff6ad-25f2-49a7-84ef-2b14c31d1ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.896002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aff6ad-25f2-49a7-84ef-2b14c31d1ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.907849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aff6ad-25f2-49a7-84ef-2b14c31d1ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:14.909891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aff6ad-25f2-49a7-84ef-2b14c31d1ce1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-a8aff6ad-25f2-49a7-84ef-2b14c31d1ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:14.926910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:14.928470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:14.929339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:14.929339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:14.929339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:14.931195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 232/462] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:14.932592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b4c5-af64-458a-a59d-3441e9de88a9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:14.933021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b4c5-af64-458a-a59d-3441e9de88a9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:14.933470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b4c5-af64-458a-a59d-3441e9de88a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:14.956690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b682b4c5-af64-458a-a59d-3441e9de88a9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:14.958494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b682b4c5-af64-458a-a59d-3441e9de88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:14.997952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:15.019343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cbdc3-fb3d-41c3-beab-8c9f3e30f436 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.057881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 233/463] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:15.058964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b4c5-af64-458a-a59d-3441e9de88a9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.070920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.071794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.071794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.072064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.072720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:15.073064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.074454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.086059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:15.100509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 231/464] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:15.106078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.184232 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b4c5-af64-458a-a59d-3441e9de88a9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.231988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feeed3a-d1a0-40ec-b18a-727b42534d7c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.238352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.239386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.239881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.240882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.242137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:15.242701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.245251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.267555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 234/465] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:15.274064 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.287629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0145d-4ed9-4bae-863a-babbe482ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.288477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0145d-4ed9-4bae-863a-babbe482ec0b None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.288897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0145d-4ed9-4bae-863a-babbe482ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.289351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0145d-4ed9-4bae-863a-babbe482ec0b None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.290149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0145d-4ed9-4bae-863a-babbe482ec0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.291886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0145d-4ed9-4bae-863a-babbe482ec0b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.314124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:15.329690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 232/466] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:25:15 2022] POST /identity/v3/users => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:15.331275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0145d-4ed9-4bae-863a-babbe482ec0b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.456524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0145d-4ed9-4bae-863a-babbe482ec0b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.502258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7c74c-21fc-43c2-8ef2-50d1f44c58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.503653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7c74c-21fc-43c2-8ef2-50d1f44c58ea None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.504078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7c74c-21fc-43c2-8ef2-50d1f44c58ea None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.504486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7c74c-21fc-43c2-8ef2-50d1f44c58ea None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.505400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7c74c-21fc-43c2-8ef2-50d1f44c58ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.508214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7c74c-21fc-43c2-8ef2-50d1f44c58ea None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.520613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:15.546591 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 233/467] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:25:15 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:15.558413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10104f-85e6-4686-a31b-e6cb939243f3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.570691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c569d2-139f-4aed-838d-b854c84aa376 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.571016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c569d2-139f-4aed-838d-b854c84aa376 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.571809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c569d2-139f-4aed-838d-b854c84aa376 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.576445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 234/468] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:15.580415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801e0c4-a9c3-4a80-ae07-b0d8c6a6efad None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.580978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801e0c4-a9c3-4a80-ae07-b0d8c6a6efad None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.581423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801e0c4-a9c3-4a80-ae07-b0d8c6a6efad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.614370 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4801e0c4-a9c3-4a80-ae07-b0d8c6a6efad None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:15.617464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4801e0c4-a9c3-4a80-ae07-b0d8c6a6efad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:15.634048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.634965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.635467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.635884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.655433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.658753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:15.676910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:15.678723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:15.680157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:15.680157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:15.680157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:15.680157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 235/469] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:15.685593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.721720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:15.754155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 235/470] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:15.785466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801e0c4-a9c3-4a80-ae07-b0d8c6a6efad None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.822682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc2573-c4e2-434c-97ce-d0cb8cf34429 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:15.862842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:15.863994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:15.864618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:15.865203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:15.867838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:15.868650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:15.870808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:15.873861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:15.897358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 236/471] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:15.902455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:15.962971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801e0c4-a9c3-4a80-ae07-b0d8c6a6efad None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:16.002689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710a858-17f9-439d-94d0-9bec0fba6b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:16.004508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710a858-17f9-439d-94d0-9bec0fba6b4f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.004938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710a858-17f9-439d-94d0-9bec0fba6b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.005524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710a858-17f9-439d-94d0-9bec0fba6b4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.017472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710a858-17f9-439d-94d0-9bec0fba6b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:16.019090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710a858-17f9-439d-94d0-9bec0fba6b4f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c710a858-17f9-439d-94d0-9bec0fba6b4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:16.035683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:16.036870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:16.038546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:16.038546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:16.038546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:16.039858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 236/472] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:16.041133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d9362-eaa0-4f7e-9b4b-c0129c476a82 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.045647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d9362-eaa0-4f7e-9b4b-c0129c476a82 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.046030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d9362-eaa0-4f7e-9b4b-c0129c476a82 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.047664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 237/473] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:16.049115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d9362-eaa0-4f7e-9b4b-c0129c476a82 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:16.051183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3439d41-2b1d-4b13-a1d1-b8c53c3bc7f5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:16.092659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9a188-f396-4990-97ca-f1dade5c5c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:16.093495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9a188-f396-4990-97ca-f1dade5c5c49 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.093869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9a188-f396-4990-97ca-f1dade5c5c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.094236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9a188-f396-4990-97ca-f1dade5c5c49 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.095258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9a188-f396-4990-97ca-f1dade5c5c49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:16.096917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9a188-f396-4990-97ca-f1dade5c5c49 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:16.098218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:16.131554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 237/474] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:25:15 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:16.131554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798aef-3726-4040-ace0-203b9ef1b8d6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.131554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798aef-3726-4040-ace0-203b9ef1b8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.131554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798aef-3726-4040-ace0-203b9ef1b8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.153494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bc798aef-3726-4040-ace0-203b9ef1b8d6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:16.157271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bc798aef-3726-4040-ace0-203b9ef1b8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:16.225671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:16.252030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d9362-eaa0-4f7e-9b4b-c0129c476a82 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:16.268240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 238/475] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:16.299986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:16.300759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.301154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.301606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.303362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:16.303764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:16.304208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc798aef-3726-4040-ace0-203b9ef1b8d6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:16.305438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:16.321262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 238/476] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:16.327143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:16.417003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc798aef-3726-4040-ace0-203b9ef1b8d6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:16.437613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:16.440469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f12031-69d3-43a2-8537-b231e1cebaad None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:16.455407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:16.456299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.456711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.457112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.458186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:16.460406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:16.474227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.sql.core [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 13:25:16.474227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 13:25:16.474227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [parameters: {'id': 'de2446f117a24bf38f3fbc5097159aaf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 13:25:16.474227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80125) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 13:25:16.481350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1187b-9d68-463e-a7ff-6c1c3e2de3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:16.482018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1187b-9d68-463e-a7ff-6c1c3e2de3a5 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.482349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1187b-9d68-463e-a7ff-6c1c3e2de3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.482878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1187b-9d68-463e-a7ff-6c1c3e2de3a5 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1187b-9d68-463e-a7ff-6c1c3e2de3a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:16.498796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1187b-9d68-463e-a7ff-6c1c3e2de3a5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result.read() Aug 10 13:25:16.498982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:25:16.499778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application yield resource Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise self.value Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application session.commit() Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 13:25:16.501152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.session.flush() Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application compat.raise_( Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 13:25:16.502273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:25:16.503152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result.read() Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:25:16.503965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [parameters: {'id': 'de2446f117a24bf38f3fbc5097159aaf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:16.504738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 13:25:16.507162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:16.509366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 239/477] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 10 13:25:16 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:16.509366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:16.528284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 239/478] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:25:16 2022] POST /identity/v3/users => generated 249 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:16.574979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:16.644422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77a32-1ce3-400d-bca0-3eaa2f6f7059 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:16.687501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86e241-0711-4ade-a487-ff0f617bb04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:16.688585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86e241-0711-4ade-a487-ff0f617bb04b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.689130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86e241-0711-4ade-a487-ff0f617bb04b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.689636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86e241-0711-4ade-a487-ff0f617bb04b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.702161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86e241-0711-4ade-a487-ff0f617bb04b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6b86e241-0711-4ade-a487-ff0f617bb04b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:16.707528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:16.708482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 13:25:16.709840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:16.713805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 240/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:16.720743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1187b-9d68-463e-a7ff-6c1c3e2de3a5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:16.853554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1187b-9d68-463e-a7ff-6c1c3e2de3a5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:16.899527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a1ef6-9e02-47c6-bee0-baffb24b0b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:16.900145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a1ef6-9e02-47c6-bee0-baffb24b0b8d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:16.900472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a1ef6-9e02-47c6-bee0-baffb24b0b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:16.900742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a1ef6-9e02-47c6-bee0-baffb24b0b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:16.901449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a1ef6-9e02-47c6-bee0-baffb24b0b8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:16.901749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a1ef6-9e02-47c6-bee0-baffb24b0b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:16.903139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a1ef6-9e02-47c6-bee0-baffb24b0b8d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:16.912311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 240/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:18.146235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb1f4-9eb3-42a9-9d72-1a436c0f8b1d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.147374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb1f4-9eb3-42a9-9d72-1a436c0f8b1d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.147655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb1f4-9eb3-42a9-9d72-1a436c0f8b1d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.149312 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 241/481] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:18.155263 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ad81-cfeb-43d6-93ce-5da8ffc38b77 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.155808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ad81-cfeb-43d6-93ce-5da8ffc38b77 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.155808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ad81-cfeb-43d6-93ce-5da8ffc38b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.184262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e957ad81-cfeb-43d6-93ce-5da8ffc38b77 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:18.185680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e957ad81-cfeb-43d6-93ce-5da8ffc38b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:18.197855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:18.281751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 241/482] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:18.300158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036dcf-f8db-44e0-a3c8-970604ec3d48 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.300573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036dcf-f8db-44e0-a3c8-970604ec3d48 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.301057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036dcf-f8db-44e0-a3c8-970604ec3d48 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.303412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 242/483] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:18.308657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648a1f-2ebd-4fd0-ae5d-68c16f6dd17a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.308936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648a1f-2ebd-4fd0-ae5d-68c16f6dd17a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.309203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648a1f-2ebd-4fd0-ae5d-68c16f6dd17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.313097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5036dcf-f8db-44e0-a3c8-970604ec3d48 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:18.342503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6f648a1f-2ebd-4fd0-ae5d-68c16f6dd17a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:18.343700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6f648a1f-2ebd-4fd0-ae5d-68c16f6dd17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:18.417656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5036dcf-f8db-44e0-a3c8-970604ec3d48 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:18.421577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:18.458589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee581ec9-194a-459e-bfc3-03afa8f3cc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:18.459351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee581ec9-194a-459e-bfc3-03afa8f3cc86 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.459686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee581ec9-194a-459e-bfc3-03afa8f3cc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.460018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee581ec9-194a-459e-bfc3-03afa8f3cc86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.470714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee581ec9-194a-459e-bfc3-03afa8f3cc86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:18.480037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 242/484] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ee581ec9-194a-459e-bfc3-03afa8f3cc86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:18.480202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:18.481045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:18.485498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 243/485] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:18.486162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f648a1f-2ebd-4fd0-ae5d-68c16f6dd17a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:18.528450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee581ec9-194a-459e-bfc3-03afa8f3cc86 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:18.618538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:18.649652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f648a1f-2ebd-4fd0-ae5d-68c16f6dd17a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:18.708213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:18.708742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.709049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.709339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.710019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:18.710312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:18.711691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:18.725110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 243/486] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:18.731656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:18.754028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee581ec9-194a-459e-bfc3-03afa8f3cc86 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:18.824540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fbc18-fd94-413e-8ebd-05de73944cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:18.825185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fbc18-fd94-413e-8ebd-05de73944cca None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.825487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fbc18-fd94-413e-8ebd-05de73944cca None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.825750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fbc18-fd94-413e-8ebd-05de73944cca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.849281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fbc18-fd94-413e-8ebd-05de73944cca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-940fbc18-fd94-413e-8ebd-05de73944cca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:18.855755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:18.856666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:18.858116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:18.863790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 244/487] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:18.867934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fbc18-fd94-413e-8ebd-05de73944cca None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:18.916282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:18.931444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cae2f1-c7bd-4876-8c2f-7ebf215e9ef0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:18.981531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390aa8e4-65b5-47ff-a896-6afe100185d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:18.982051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390aa8e4-65b5-47ff-a896-6afe100185d8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:18.982325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390aa8e4-65b5-47ff-a896-6afe100185d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:18.982836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390aa8e4-65b5-47ff-a896-6afe100185d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:18.993945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390aa8e4-65b5-47ff-a896-6afe100185d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:18.995435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390aa8e4-65b5-47ff-a896-6afe100185d8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:18.998672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-390aa8e4-65b5-47ff-a896-6afe100185d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:19.011599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:19.013665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:19.015762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:19.015762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:19.015762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:19.019984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 244/488] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:19.023216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b673b-671c-4c19-b2dd-f50a2e056b4b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.023904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b673b-671c-4c19-b2dd-f50a2e056b4b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.024614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b673b-671c-4c19-b2dd-f50a2e056b4b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.027627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 245/489] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:19.029573 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b673b-671c-4c19-b2dd-f50a2e056b4b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.047623 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fbc18-fd94-413e-8ebd-05de73944cca None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:19.099365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030c8a3-5cdd-4393-8458-253d20e7575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:19.101337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030c8a3-5cdd-4393-8458-253d20e7575a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.102009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030c8a3-5cdd-4393-8458-253d20e7575a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.102693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030c8a3-5cdd-4393-8458-253d20e7575a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.104107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030c8a3-5cdd-4393-8458-253d20e7575a None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:19.104842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030c8a3-5cdd-4393-8458-253d20e7575a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:19.107988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030c8a3-5cdd-4393-8458-253d20e7575a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:19.123655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 245/490] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:19.126161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb977-a91f-41cc-9705-e39466b71d73 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.126883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb977-a91f-41cc-9705-e39466b71d73 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.127860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb977-a91f-41cc-9705-e39466b71d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.159717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-44dbb977-a91f-41cc-9705-e39466b71d73 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:19.163717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-44dbb977-a91f-41cc-9705-e39466b71d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:19.171713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b673b-671c-4c19-b2dd-f50a2e056b4b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:19.220794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:19.222174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.222896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.223588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.225012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:19.225801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:19.228530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:19.249415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 246/491] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:19.250702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.297280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 246/492] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:19.297280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbb977-a91f-41cc-9705-e39466b71d73 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.310771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:19.320008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:19.393882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76453ea4-2094-4428-819d-067d6bbb8dc5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:19.410836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbb977-a91f-41cc-9705-e39466b71d73 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:19.446337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:19.446701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0951c8-431d-4b8c-8b37-4f4b6e3fd818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:19.446965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.447235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.447328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0951c8-431d-4b8c-8b37-4f4b6e3fd818 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.447572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.447665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0951c8-431d-4b8c-8b37-4f4b6e3fd818 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.447841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0951c8-431d-4b8c-8b37-4f4b6e3fd818 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.455504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0951c8-431d-4b8c-8b37-4f4b6e3fd818 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:19.456977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0951c8-431d-4b8c-8b37-4f4b6e3fd818 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:19.458237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:19.460328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-db0951c8-431d-4b8c-8b37-4f4b6e3fd818 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:19.469927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:19.470829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:19.471732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 247/493] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:19.472306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e578-e12f-4ad7-b7de-3218ef33743a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.472988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e578-e12f-4ad7-b7de-3218ef33743a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.472988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e578-e12f-4ad7-b7de-3218ef33743a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.477469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 248/494] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:19.477469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7e578-e12f-4ad7-b7de-3218ef33743a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:19.482529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:19.483648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:19.485192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:19.485192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:19.485192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:19.485192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 247/495] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:19.485192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.574378 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:19.599986 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7e578-e12f-4ad7-b7de-3218ef33743a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:19.617440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:19.621711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634493b-4f01-42a5-9be9-6429f744ab7c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:19.638070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edbd9b-baca-4e1d-9490-4f378fde0d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:19.638711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edbd9b-baca-4e1d-9490-4f378fde0d9c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.638983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edbd9b-baca-4e1d-9490-4f378fde0d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.639251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edbd9b-baca-4e1d-9490-4f378fde0d9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.650010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edbd9b-baca-4e1d-9490-4f378fde0d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d6edbd9b-baca-4e1d-9490-4f378fde0d9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:19.655009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:19.655907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:19.656704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 249/496] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:19.657997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e8112-5d78-41ab-8e55-f48eba364fcd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.657997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e8112-5d78-41ab-8e55-f48eba364fcd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.657997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e8112-5d78-41ab-8e55-f48eba364fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.665982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:19.666600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.666881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.667153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.667896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:19.668197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:19.669703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:19.680001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-527e8112-5d78-41ab-8e55-f48eba364fcd None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:19.681543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-527e8112-5d78-41ab-8e55-f48eba364fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:19.709409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 248/497] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:19.710335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.754841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 250/498] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:19.756508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e8112-5d78-41ab-8e55-f48eba364fcd None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.815131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89748233-689e-480b-87e4-4df03f1cd549 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:19.827678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:19.848898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5524961-9710-455f-9788-b4fc9b791fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:19.849546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5524961-9710-455f-9788-b4fc9b791fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.849824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5524961-9710-455f-9788-b4fc9b791fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.850098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5524961-9710-455f-9788-b4fc9b791fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.850740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5524961-9710-455f-9788-b4fc9b791fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:19.851131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5524961-9710-455f-9788-b4fc9b791fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:19.852511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5524961-9710-455f-9788-b4fc9b791fe5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:19.862222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e8112-5d78-41ab-8e55-f48eba364fcd None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:19.864120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 249/499] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:19.865946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfeb99-26ec-4a12-b58c-31146082a2da None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.866221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfeb99-26ec-4a12-b58c-31146082a2da None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.866492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfeb99-26ec-4a12-b58c-31146082a2da None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.867697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 250/500] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:19.868388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfeb99-26ec-4a12-b58c-31146082a2da None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.887359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:19.902076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:19.902692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:19.902970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:19.903241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:19.903900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:19.904209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:19.905614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:19.917975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 251/501] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:19.917975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:19.973795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfeb99-26ec-4a12-b58c-31146082a2da None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.008001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4813bd9-e6a3-416a-b749-d4747b35eecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.008601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4813bd9-e6a3-416a-b749-d4747b35eecf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.008882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4813bd9-e6a3-416a-b749-d4747b35eecf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.009151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4813bd9-e6a3-416a-b749-d4747b35eecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.012617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cfb36-d48f-45db-b8b0-2b1b187e59ae None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.039347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.047365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae492c9c-0db1-442c-add0-34ee1435e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.047999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae492c9c-0db1-442c-add0-34ee1435e485 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.048276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae492c9c-0db1-442c-add0-34ee1435e485 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.048542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae492c9c-0db1-442c-add0-34ee1435e485 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.060536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae492c9c-0db1-442c-add0-34ee1435e485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ae492c9c-0db1-442c-add0-34ee1435e485 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:20.065812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:20.066871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:20.067848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:20.072532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 252/502] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:20.073455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae492c9c-0db1-442c-add0-34ee1435e485 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.080844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4813bd9-e6a3-416a-b749-d4747b35eecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.082278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4813bd9-e6a3-416a-b749-d4747b35eecf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.100859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.173690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae492c9c-0db1-442c-add0-34ee1435e485 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.190610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 251/503] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:20.191877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc722f33-904b-4f1f-9385-a79aaf38c7f9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.192161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc722f33-904b-4f1f-9385-a79aaf38c7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.192480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc722f33-904b-4f1f-9385-a79aaf38c7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.212122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-dc722f33-904b-4f1f-9385-a79aaf38c7f9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:20.213352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dc722f33-904b-4f1f-9385-a79aaf38c7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:20.213458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.214080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.214356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.214622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.222317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.223753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:20.235614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:20.236704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 253/504] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:20.237674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.288501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 252/505] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:20.289386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc722f33-904b-4f1f-9385-a79aaf38c7f9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.300459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.342514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.359248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ddfe9-468f-4d16-a7fe-3e1486dae3d9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.394226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.394860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.395136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.395404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.395517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc722f33-904b-4f1f-9385-a79aaf38c7f9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.408337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.409955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:20.424654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:20.425651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:20.426470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:20.426470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:20.426470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:20.426470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 254/506] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:20.426690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.441017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.441639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.441915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.442184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.442847 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:20.443148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.444549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.452981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 253/507] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:20.453877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.531748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.535822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ed15-30fd-49a5-bf7c-f63eda2c6f04 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.570144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0bd0a-5928-4f19-83a6-acda67d771db None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.571124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.571758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.572030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.572294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.580826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.583492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.585719 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:20.600368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:20.601561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:20.602394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:20.602394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:20.602394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:20.602394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 255/508] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:20.602394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.610579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.611619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.611908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.612177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.612913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:20.613590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.615841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.634683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 254/509] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:20.635369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.702293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e5dae-79ae-4ead-8f62-a278c797e04f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.733869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e595c73-7b50-4b46-9af9-594a5934fc40 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.735068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.735699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.735973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.736248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.747030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.748468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:20.763177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:20.764081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:20.764923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:20.764923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:20.764923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:20.764923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 256/510] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:20.765129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.768571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.773582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.774187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.774459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.774755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.775391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:20.775684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.777049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.788368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 255/511] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:20.789150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.820841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:20.864485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb520c7-eb87-4d67-9c42-c5887ecc0155 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.893553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f85d90-6313-41cb-9e40-b6f816d4c375 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:20.897799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.898432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.898707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.898974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.909639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.911106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:20.925824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:20.926987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:20.928412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:20.928412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:20.928412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:20.928412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 257/512] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:20.928412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.932041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:20.932508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:20.932784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:20.933048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:20.933718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:20.934034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:20.935425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:20.951583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 256/513] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:20.952411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:20.992224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:21.041340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5e185-ae61-47b5-9ea2-ad51266223ea None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.046706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:21.060037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3efab7-1c6c-42de-a07d-4d4f684dde70 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.088813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda46a8f-58fc-4d8b-be7a-624ed75b3909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.089456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda46a8f-58fc-4d8b-be7a-624ed75b3909 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.089732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda46a8f-58fc-4d8b-be7a-624ed75b3909 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.089999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda46a8f-58fc-4d8b-be7a-624ed75b3909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.097197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.097821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.098092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.098356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.099898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:21.100393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.102052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.114322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 257/514] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:21.115248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.165679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda46a8f-58fc-4d8b-be7a-624ed75b3909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.167223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda46a8f-58fc-4d8b-be7a-624ed75b3909 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.217178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ebd76-340c-4a90-a2a7-f4c533571d0e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.221630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:21.250927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.251548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.251817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.252078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.259761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.261217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.274210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 258/515] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:21.274210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda46a8f-58fc-4d8b-be7a-624ed75b3909 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:21.274458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:21.275257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:21.276139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 258/516] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:21.276707 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.288285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:21.379842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda46a8f-58fc-4d8b-be7a-624ed75b3909 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.390128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c51f0-2f00-4ad8-9d7e-241e04bd846c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.414357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.414979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.415257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.415522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.416163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:21.416489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.417911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.424210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.425065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.425379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.425654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.429763 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 259/517] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:21.430033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.434528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.436361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:21.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:21.456945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 259/518] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:21.458629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.464776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:21.527760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:21.573736 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cdcd1-b3b6-4731-b352-99e310f1890f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.573949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0def95c-6e4a-426c-99dc-1a32a072e880 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.610673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.611301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.611581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.611855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.619211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa2edd-6a85-41e4-946d-f65eab711ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.619813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa2edd-6a85-41e4-946d-f65eab711ccf None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.620085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa2edd-6a85-41e4-946d-f65eab711ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.620355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa2edd-6a85-41e4-946d-f65eab711ccf None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.620972 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa2edd-6a85-41e4-946d-f65eab711ccf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.622242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.622420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa2edd-6a85-41e4-946d-f65eab711ccf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.623691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:21.638294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:21.640070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:21.641775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:21.641775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:21.641775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:21.641775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 260/519] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:21.641775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.654844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 260/520] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:25:21 2022] POST /identity/v3/users => generated 247 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:21.656612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa2edd-6a85-41e4-946d-f65eab711ccf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.758931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:21.768746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb75e6a-ce12-4116-9e61-a70a38163d81 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.795526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa2edd-6a85-41e4-946d-f65eab711ccf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.807362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:21.831056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.831602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.831874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.832150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.832880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:21.833174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.835215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.839816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:21.840678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:21.840951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:21.841215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:21.851299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:21.852765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:21.855970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 261/521] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:21.857064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:21.869642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:21.871032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:25:21.872586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:21.876192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 261/522] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:21.877544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:21.975653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e69111-dd21-4a63-9f18-3265cd310778 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:21.994507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080db05-4e2c-4ce8-925c-9386dacc8c26 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.012335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.012935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.013205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.013497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.014178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:22.014474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.015915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.024496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.030667 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.031278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.031552 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.031811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.032443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:22.032738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.034267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 262/523] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:22.034374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.038235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:22.049587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 262/524] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:22.050638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:22.083219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.150632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78c13-f6d9-444a-825b-a66e0013bb2a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.156673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b28725-c0d0-4ae8-93be-f416c7fb8b8c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.188254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83d11e-6959-4ac8-9e82-c47d93d7e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.188849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83d11e-6959-4ac8-9e82-c47d93d7e751 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.189122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83d11e-6959-4ac8-9e82-c47d93d7e751 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.189426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83d11e-6959-4ac8-9e82-c47d93d7e751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.199168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.200133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.200582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.201021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.203375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:22.203375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.205187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.253153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 263/525] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:22.254000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:22.280556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83d11e-6959-4ac8-9e82-c47d93d7e751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.282221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83d11e-6959-4ac8-9e82-c47d93d7e751 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.309206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.327730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.361930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd32631-55b7-49e6-9f15-14a8d293fa2b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.396384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 263/526] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:22.401465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83d11e-6959-4ac8-9e82-c47d93d7e751 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:22.404339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.405065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.405442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.405773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.406615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=ce851cf0509947e783dbd5b4809bc897) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:22.406979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.408492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.417655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 264/527] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/role_assignments?user.id=ce851cf0509947e783dbd5b4809bc897&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:22.418481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:22.548936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83d11e-6959-4ac8-9e82-c47d93d7e751 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.556382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a645-699a-453a-b9ad-5f01fde92036 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.577346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.598506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.599134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.599466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.599767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.609142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.610647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.611432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51965fa5-20ad-43aa-a634-a65be361d0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.612528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.612901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51965fa5-20ad-43aa-a634-a65be361d0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.614375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51965fa5-20ad-43aa-a634-a65be361d0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.614864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51965fa5-20ad-43aa-a634-a65be361d0a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:22.624067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:22.625058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:22.625936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 264/528] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:22.626969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:22.693777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51965fa5-20ad-43aa-a634-a65be361d0a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.695514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51965fa5-20ad-43aa-a634-a65be361d0a1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.732935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8684e4-4407-4b93-bef7-e19cf9c70ac2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.770136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.770940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.771530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.771831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.782825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.784417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:22.806421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:22.807368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:22.808217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:22.808217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:22.808217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:22.808217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 265/529] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:22.808924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:22.846155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 265/530] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:22.847119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51965fa5-20ad-43aa-a634-a65be361d0a1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:22.868071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.871694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:22.930192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572476e5-2991-4da5-aba4-bf40226e8f30 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.967850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:22.969085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:22.969581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:22.970040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:22.970990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:22.971506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:22.971775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51965fa5-20ad-43aa-a634-a65be361d0a1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:22.973465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:22.988772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 266/531] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:22.991802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.021281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.022061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.022338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.022608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.023240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:23.023553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.025676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.041200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 266/532] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:23.042326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.107048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0651cb-277a-43e9-89e0-7245a7c45413 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.121271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:23.139325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:23.151481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.152473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.152966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.153460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.168280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.169011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33729e-5fdb-49ca-9911-e6c35619328b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.169797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:23.189210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:23.190198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:23.191042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:23.191042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:23.191042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:23.191042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 267/533] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:23.191670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.209541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.210796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.211830 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.212564 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.213846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=4e395e059b5d436a8678bb302bcfafb5) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:23.214352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.216379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.224240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 267/534] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/role_assignments?user.id=4e395e059b5d436a8678bb302bcfafb5&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:23.225952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.374607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f113c-a881-432e-bea9-93a66e24157e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.401134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa48ee-08bd-4927-88b2-36d8caeb348a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.449710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.449934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.450139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.450384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.458167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.459331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.459928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.460402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.460789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.461084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.461966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:23.462450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.464035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:23.473869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:23.475562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:23.476598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:23.476598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:23.476598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:23.476598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:23.476598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:23.476598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:23.476598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:23.476598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:23.480298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 268/535] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:23.480772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 268/536] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:23.485260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:23.485260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.490199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.533706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:23.674137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd11274-6f7a-482b-b118-e780869889de None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.677451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494a27c-0cf9-4739-a27a-3f8fbcae7f4a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.735803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.736419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.736695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.736964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.737948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.738561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.738830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.739090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.740593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:23.740896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.742372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.760000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.761561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:23.776947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:23.777894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:23.778715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 269/537] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:23.785001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.787532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 269/538] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:23.793068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:23.864188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:23.912928 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:23.933258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c3d4f-5ed9-4bc8-ab60-5dde25e855c8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.943780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c0f43-223b-4e1a-a4c7-1a6f35360f3c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:23.967419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80d86b-6966-41f9-98b5-2f3d9d263baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:23.968019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80d86b-6966-41f9-98b5-2f3d9d263baa None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.968292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80d86b-6966-41f9-98b5-2f3d9d263baa None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.968556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80d86b-6966-41f9-98b5-2f3d9d263baa None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.969255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80d86b-6966-41f9-98b5-2f3d9d263baa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:23.969566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80d86b-6966-41f9-98b5-2f3d9d263baa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:23.970995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80d86b-6966-41f9-98b5-2f3d9d263baa None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:23.984195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 270/539] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:23.990760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605260-0363-4ed5-a3b2-d6de5eb5ae9d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:23.991076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605260-0363-4ed5-a3b2-d6de5eb5ae9d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:23.991350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605260-0363-4ed5-a3b2-d6de5eb5ae9d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:23.992934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 271/540] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:23.993710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3605260-0363-4ed5-a3b2-d6de5eb5ae9d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:24.016180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7acf8-e58a-45b2-be3d-7b62714668c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:24.017338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7acf8-e58a-45b2-be3d-7b62714668c1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.017838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7acf8-e58a-45b2-be3d-7b62714668c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.018326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7acf8-e58a-45b2-be3d-7b62714668c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.019700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7acf8-e58a-45b2-be3d-7b62714668c1 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=b96be241231247b39e544529a4cd0b74) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:24.020334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7acf8-e58a-45b2-be3d-7b62714668c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:24.023165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7acf8-e58a-45b2-be3d-7b62714668c1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:24.034138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 270/541] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/role_assignments?user.id=b96be241231247b39e544529a4cd0b74&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:24.036293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a224a-a69e-4dd5-b384-a698025ec00d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.037470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a224a-a69e-4dd5-b384-a698025ec00d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.037963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a224a-a69e-4dd5-b384-a698025ec00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.079531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c92a224a-a69e-4dd5-b384-a698025ec00d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:24.082128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c92a224a-a69e-4dd5-b384-a698025ec00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:24.160049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3605260-0363-4ed5-a3b2-d6de5eb5ae9d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:24.199819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:24.200343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.200617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.200884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.201665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=234eed198ab84e95ae1a7f03785a1481) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:24.201962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:24.203476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:24.210982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 272/542] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:23 2022] GET /identity/v3/role_assignments?user.id=234eed198ab84e95ae1a7f03785a1481&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:24.216134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 271/543] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:24.252388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:24.270418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:24.416234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceb418-34d4-4328-b918-6bc2bb406a0b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:24.456822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787fa96-1b2d-447f-8619-4f9a6b77f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:24.457407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fa96-1b2d-447f-8619-4f9a6b77f922 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.457684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fa96-1b2d-447f-8619-4f9a6b77f922 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.457951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fa96-1b2d-447f-8619-4f9a6b77f922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.473912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787fa96-1b2d-447f-8619-4f9a6b77f922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b787fa96-1b2d-447f-8619-4f9a6b77f922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:24.480395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:24.481294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:24.482084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:24.483375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 273/544] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:24.490409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a224a-a69e-4dd5-b384-a698025ec00d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:24.572285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:24.611280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2513-2fa7-434e-acbb-2df8e8dc8455 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.611732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2513-2fa7-434e-acbb-2df8e8dc8455 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.612115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2513-2fa7-434e-acbb-2df8e8dc8455 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.613910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 274/545] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:24.622776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27330f-9e57-4d9f-a65f-e516e017c5bb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.623161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27330f-9e57-4d9f-a65f-e516e017c5bb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.623587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27330f-9e57-4d9f-a65f-e516e017c5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.624794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a224a-a69e-4dd5-b384-a698025ec00d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:24.663831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:24.664696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.665138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.665272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-af27330f-9e57-4d9f-a65f-e516e017c5bb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:24.665666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.666468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:24.666735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-af27330f-9e57-4d9f-a65f-e516e017c5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:24.667070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:24.668566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:24.678452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 272/546] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:24.684524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:24.782398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:24.790325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 275/547] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:24.794311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b6acb-bf7d-4725-8ea3-02f2039dd94d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:24.808436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:24.814907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27330f-9e57-4d9f-a65f-e516e017c5bb None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:24.832528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:24.832528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:24.832528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:24.832528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:24.842551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:24.844008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:24.860141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:24.861336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:24.862405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:24.862405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:24.862405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:24.862405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 273/548] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:24.867421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:24.990500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27330f-9e57-4d9f-a65f-e516e017c5bb None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:25.031482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62d070-d1fb-4ba8-b788-98112c20de91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:25.034902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62d070-d1fb-4ba8-b788-98112c20de91 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.035127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62d070-d1fb-4ba8-b788-98112c20de91 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.035395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62d070-d1fb-4ba8-b788-98112c20de91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.061577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62d070-d1fb-4ba8-b788-98112c20de91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-fa62d070-d1fb-4ba8-b788-98112c20de91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:25.079224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:25.080205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:25.081528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 276/549] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:25.091032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c67fb-7234-4c99-b234-bab1ed6522f8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:25.094284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62d070-d1fb-4ba8-b788-98112c20de91 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:25.129871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:25.130495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.130807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.131079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.131723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:25.132024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:25.133435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:25.147871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 274/550] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:25.153961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:25.209614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:25.227168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:25.311233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62d070-d1fb-4ba8-b788-98112c20de91 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:25.321524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ffa34-4dda-40a8-8bfa-5f4ca8797ba4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:25.362066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:25.362705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.362978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.363244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] PATH_INFO: `/v3/users/swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.372282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:25.373918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:25.382337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:25.383241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.383719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.384186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.385054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:25.385622 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:25.387043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:25.388005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:25.389022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:25.390283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:25.397531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 275/551] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:25.397531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:25.406928 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 277/552] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:25.413742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:25.569900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:25.590391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d281b64-8240-40a4-89ac-8233bca7f7af None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:25.634387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:25.636252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.636747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.637224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.638196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:25.638723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:25.640389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:25.663807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 276/553] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 10 13:25:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:25.667362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bd949-5f47-42b9-ac3d-8be223268dd0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:25.669251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:25.722695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:25.786845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8c962-8516-47d1-a9d7-837469d906b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:25.788203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8c962-8516-47d1-a9d7-837469d906b7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.788481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8c962-8516-47d1-a9d7-837469d906b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.788760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8c962-8516-47d1-a9d7-837469d906b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.793833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d5339-75b4-42d8-989d-3fc2d60eb718 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:25.801598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8c962-8516-47d1-a9d7-837469d906b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:25.803751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8c962-8516-47d1-a9d7-837469d906b7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-91b8c962-8516-47d1-a9d7-837469d906b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:25.822920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:25.823970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:25.824853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:25.824853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:25.824853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:25.824853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 278/554] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:25.830556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc83fec-8073-4066-914f-df958b28d64e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.830835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc83fec-8073-4066-914f-df958b28d64e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.831106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc83fec-8073-4066-914f-df958b28d64e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.833048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 279/555] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:25.833927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc83fec-8073-4066-914f-df958b28d64e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:25.842561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5418e-34a8-49af-bcb4-2cacf7c88a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:25.843456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5418e-34a8-49af-bcb4-2cacf7c88a21 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:25.843938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5418e-34a8-49af-bcb4-2cacf7c88a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:25.844881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5418e-34a8-49af-bcb4-2cacf7c88a21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:25.866551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:25.936877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5418e-34a8-49af-bcb4-2cacf7c88a21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:25.938972 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5418e-34a8-49af-bcb4-2cacf7c88a21 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:25.967577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc83fec-8073-4066-914f-df958b28d64e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.007329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987480b-2b1b-464a-a83c-5f40dba4b22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.008459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987480b-2b1b-464a-a83c-5f40dba4b22e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.009082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987480b-2b1b-464a-a83c-5f40dba4b22e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.009608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987480b-2b1b-464a-a83c-5f40dba4b22e None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.010460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987480b-2b1b-464a-a83c-5f40dba4b22e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:26.010959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987480b-2b1b-464a-a83c-5f40dba4b22e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.012663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987480b-2b1b-464a-a83c-5f40dba4b22e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.029022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 280/556] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:26.030729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9bbef-652c-4115-9d77-b7ed99a9fa4a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.031216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9bbef-652c-4115-9d77-b7ed99a9fa4a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.032015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9bbef-652c-4115-9d77-b7ed99a9fa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.053030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-51e9bbef-652c-4115-9d77-b7ed99a9fa4a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:26.054719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-51e9bbef-652c-4115-9d77-b7ed99a9fa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:26.060306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 277/557] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:25:26.061264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5418e-34a8-49af-bcb4-2cacf7c88a21 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.087756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.109555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.142859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 281/558] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:26.144147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9bbef-652c-4115-9d77-b7ed99a9fa4a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.168660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5418e-34a8-49af-bcb4-2cacf7c88a21 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.205523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.206197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.206519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.206791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.216109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.218370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:26.235346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:26.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:26.237795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 278/559] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:26.241934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.278607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9bbef-652c-4115-9d77-b7ed99a9fa4a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.319536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.320621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.321101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.321702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.335069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.337039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:26.354498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:26.355570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:26.356455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:26.356455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:26.356455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:26.359170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 282/560] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:26.360235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.366706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2ee23-7416-4ead-ad06-6ddc5f64259e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.370217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.380415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.416855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05567803-944d-4bae-b89f-338852d1cee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.417967 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05567803-944d-4bae-b89f-338852d1cee5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.418799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05567803-944d-4bae-b89f-338852d1cee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.419194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05567803-944d-4bae-b89f-338852d1cee5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.434437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05567803-944d-4bae-b89f-338852d1cee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-05567803-944d-4bae-b89f-338852d1cee5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:26.440498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:26.441702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:26.442761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:26.445053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 279/561] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:26.446240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05567803-944d-4bae-b89f-338852d1cee5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.484241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6167acf-d466-439c-b8e9-45648917ce6c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.527424 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.528633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.529320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.529847 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.531043 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:26.531542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.534261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.560192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 283/562] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:26.560192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.578146 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05567803-944d-4bae-b89f-338852d1cee5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.615211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a1c2b-3040-4626-8f66-6939085c26fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.615729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a1c2b-3040-4626-8f66-6939085c26fa None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.615999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a1c2b-3040-4626-8f66-6939085c26fa None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.616261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a1c2b-3040-4626-8f66-6939085c26fa None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.616892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a1c2b-3040-4626-8f66-6939085c26fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:26.617188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a1c2b-3040-4626-8f66-6939085c26fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.618558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a1c2b-3040-4626-8f66-6939085c26fa None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.633270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 280/563] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:26.633695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bc150-1604-4195-8b09-94a746d1d9b6 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.634308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bc150-1604-4195-8b09-94a746d1d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.634587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bc150-1604-4195-8b09-94a746d1d9b6 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.635798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 281/564] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:26.636488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bc150-1604-4195-8b09-94a746d1d9b6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.642660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.661890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.674430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc1d15-d8ff-43ce-a4ab-3496d4cc6007 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.707007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993a88d-2539-4693-aa8d-5499ebeda60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.707602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993a88d-2539-4693-aa8d-5499ebeda60c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.707877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993a88d-2539-4693-aa8d-5499ebeda60c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.708144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993a88d-2539-4693-aa8d-5499ebeda60c None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.709341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993a88d-2539-4693-aa8d-5499ebeda60c None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:26.709641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993a88d-2539-4693-aa8d-5499ebeda60c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.711022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993a88d-2539-4693-aa8d-5499ebeda60c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.718768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 284/565] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:26.719924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98d08-62d1-4e53-9ccd-ed9e88dcb03b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.720203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98d08-62d1-4e53-9ccd-ed9e88dcb03b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.720470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98d08-62d1-4e53-9ccd-ed9e88dcb03b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.721899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 285/566] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:26.726042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98d08-62d1-4e53-9ccd-ed9e88dcb03b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.741964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bc150-1604-4195-8b09-94a746d1d9b6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.776110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a9b92-0311-4855-a45c-75d7bba3535a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.776737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9b92-0311-4855-a45c-75d7bba3535a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.777015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9b92-0311-4855-a45c-75d7bba3535a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.777357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9b92-0311-4855-a45c-75d7bba3535a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.787675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a9b92-0311-4855-a45c-75d7bba3535a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.789125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a9b92-0311-4855-a45c-75d7bba3535a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d89a9b92-0311-4855-a45c-75d7bba3535a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:26.803562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:26.804670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:26.805604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:26.805604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:26.805604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:26.805604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 282/567] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:26.806106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23774380-3c3e-4fdc-808c-510877a65147 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.806106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23774380-3c3e-4fdc-808c-510877a65147 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.806358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23774380-3c3e-4fdc-808c-510877a65147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.825793 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-23774380-3c3e-4fdc-808c-510877a65147 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:26.827014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-23774380-3c3e-4fdc-808c-510877a65147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:26.838612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98d08-62d1-4e53-9ccd-ed9e88dcb03b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:26.868424 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.873788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:26.874409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.874686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.874956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.881603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:26.883045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:26.893927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:26.894802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:26.895680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 286/568] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:26.896413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:26.901613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 283/569] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:26.902850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49273bcb-7922-461e-9290-efe74aa5be9a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:26.903132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49273bcb-7922-461e-9290-efe74aa5be9a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:26.903405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49273bcb-7922-461e-9290-efe74aa5be9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:26.918138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:26.923798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-49273bcb-7922-461e-9290-efe74aa5be9a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:26.925374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-49273bcb-7922-461e-9290-efe74aa5be9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:26.991176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 284/570] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:26.992013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49273bcb-7922-461e-9290-efe74aa5be9a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.003534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b818-b4f5-44ad-b37d-edf4ec839c0a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.037841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.038459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.038741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.039007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.049298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.050754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:27.065023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:27.066102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:27.066984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:27.066984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:27.066984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:27.066984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 287/571] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:27.066984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.096564 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49273bcb-7922-461e-9290-efe74aa5be9a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.117445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.133316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.133928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.134205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.134477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.135159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:27.135423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.136785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.139374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.149989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 285/572] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:27.150818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.180803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d70ca-a57c-4f44-b98f-96fa57ab2947 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.216554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.217158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.217469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.217741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.218436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:27.218747 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.220182 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.237323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 288/573] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:27.238699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.266626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda70eec-c538-43ec-9f7f-1d42f473a21f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.326886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737712d9-702f-4944-a9a7-b27d9984b1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.327891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737712d9-702f-4944-a9a7-b27d9984b1aa None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.328358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737712d9-702f-4944-a9a7-b27d9984b1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.328813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737712d9-702f-4944-a9a7-b27d9984b1aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.345350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737712d9-702f-4944-a9a7-b27d9984b1aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-737712d9-702f-4944-a9a7-b27d9984b1aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:27.351062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:27.352716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 286/574] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:27.354614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737712d9-702f-4944-a9a7-b27d9984b1aa None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.378258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115b3d2-b989-494e-8b84-c21b37dec6cb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.381963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.427202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f0e8d-9416-4949-95d6-7124d14c268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.427793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f0e8d-9416-4949-95d6-7124d14c268d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.428068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f0e8d-9416-4949-95d6-7124d14c268d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.428338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f0e8d-9416-4949-95d6-7124d14c268d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.430968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.439443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f0e8d-9416-4949-95d6-7124d14c268d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bc8f0e8d-9416-4949-95d6-7124d14c268d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:27.444101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:27.445090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 289/575] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:27.446799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f0e8d-9416-4949-95d6-7124d14c268d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.482087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737712d9-702f-4944-a9a7-b27d9984b1aa None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.517152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.517792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.518063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.518335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.518976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:27.519274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.520634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.533080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 287/576] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:27.533944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.559833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f0e8d-9416-4949-95d6-7124d14c268d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.597675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.598409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.598683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.598947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.607485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.609195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:27.620482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:27.621512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 290/577] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:27.622394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.646792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059e6d1-2380-455c-a582-b22a5691996d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.651303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.664757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.685473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.686343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.686822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.687227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.688329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=10209fea1bcc43509534b7e20f635c75) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:27.688777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.691015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.708004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 288/578] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/role_assignments?user.id=10209fea1bcc43509534b7e20f635c75&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:27.708004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.745724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91ab5a-3d26-4275-b4d2-2cc396b4a65a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.787251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.787872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.788150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.788417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.789112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:27.789447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.790869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.807557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 291/579] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:27.807557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.839966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9e96c-902c-4bc6-a2bc-5e8caf6cbe1d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.883339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.884407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.884832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.885266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.886090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:27.886564 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.888405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.898958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 289/580] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:27.899828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:27.911226 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.930093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3e599-0f6f-4279-8012-354f1583eb4f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:27.951106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:27.970019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:27.971225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:27.971678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:27.972128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:27.984577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:27.986125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:27.998937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:28.000030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 292/581] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:28.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.024950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b5cf-f6cc-4de1-8958-ff3d1cc5ecb4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.066948 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.067551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.067832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.068709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.069422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:28.069755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.071255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.090688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 290/582] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:28.090688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.126505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e944dd-eb56-4d2c-9843-0aa9a0340c00 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.163444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.164896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.165178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.165503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.179997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.181518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.188613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:28.196197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d9903-1af4-46f1-bc43-e6b914053c4f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.197916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:28.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:28.201589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:28.208222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:28.208222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:28.208222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:28.208222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 293/583] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:28.208222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.238309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.238940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.239215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.239480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.255327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.257174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:28.274003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:28.274968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:28.276509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:28.276509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:28.276509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:28.276509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 291/584] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:28.276509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.362096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ca11a-aabc-4a09-b74e-d93fe1f693bd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.385534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6536c6e-75b1-41f9-b374-7bcbb94577eb None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.406574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.407170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.407683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.407851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.408570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:28.408865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.410326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.433642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 294/585] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:28.433642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.446177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408d28-1f3d-4eed-8f6c-40e810896162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.446870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408d28-1f3d-4eed-8f6c-40e810896162 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.447272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408d28-1f3d-4eed-8f6c-40e810896162 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.447542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408d28-1f3d-4eed-8f6c-40e810896162 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/ce851cf0509947e783dbd5b4809bc897/roles/a9f22cef53e740cdbc09d1137362031e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.471233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca408d28-1f3d-4eed-8f6c-40e810896162 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=ce851cf0509947e783dbd5b4809bc897, role_id=a9f22cef53e740cdbc09d1137362031e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.473678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca408d28-1f3d-4eed-8f6c-40e810896162 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.505262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 292/586] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:25:28 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/ce851cf0509947e783dbd5b4809bc897/roles/a9f22cef53e740cdbc09d1137362031e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:28.505262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408d28-1f3d-4eed-8f6c-40e810896162 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.508173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:28.518008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:28.570115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726e6fd-921e-439a-9b82-489ccb78c2e4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.611682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.612725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.613210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.613728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.614616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:28.615689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.617423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.631906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408d28-1f3d-4eed-8f6c-40e810896162 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.632593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 295/587] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:28.633676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.682716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.683350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.684057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.684057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.684479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:28.684776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.686188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.700115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 293/588] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:28.701082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.775412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:28.776029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4efb2-6267-4985-9663-5deb3350e262 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.805188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:28.822962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a634c-56ed-4698-9541-f286f914f1ea None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:28.831296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.832833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.833114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.833422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.850926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.853141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.864171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:28.865551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:28.865829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:28.866088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:28.875947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:28.876929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:28.877851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:28.877851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:28.877851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:28.878020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:28.878020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:28.880074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 296/589] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:28.881751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:28.892026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:28.893252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:25:28.894358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:28.899516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 294/590] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:28.900495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:28.996466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549fb98-69ad-4184-a6ef-4e68c66404f3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.014528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074feac-f994-444f-8af0-4cbe7396e2c5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.043127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:29.045633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.046651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.047152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.047597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.056606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.057306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.057586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.057851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.058496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:29.058789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.060148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.060669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.062692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.074081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 295/591] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:29.075172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:29.076740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:29.077690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:25:29.078629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:29.085426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 297/592] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:29.086726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.119575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:29.188407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eff6a9-2698-4fac-b719-98363ae28b39 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.198151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893906c-461c-45d1-a5fd-f2a20e932203 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.224784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.225412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.225684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.225948 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.226664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:29.226959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.228577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.236813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.237847 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.238394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.238889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.247046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.249364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.258254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 296/593] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:29.259158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:29.263503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:29.264423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:29.265336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:29.265336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:29.265336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:29.265336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:29.265336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:29.265336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:29.265336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:29.265336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:29.267761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 298/594] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:29.268901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.307504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:29.365589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ce9de-dc5c-40d9-ad50-ea9aa4a35fc6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.377628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:29.382104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf4b17-03e7-4eef-820c-3ac3eb90fab3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.400529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.401140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.401440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.401705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.402445 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:29.402715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.404122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.420895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.425262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 297/595] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:29.425262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.425820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.426413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.426930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.440154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.442222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:29.458951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:29.459915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:29.460782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:29.460782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:29.460782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:29.462836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 299/596] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:29.463835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.525771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:29.528525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99786-a6b5-4230-ac4a-f717677da087 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.565364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.566072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.566474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.566819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.567585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:29.568017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.570581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.577287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33c476-a4bc-4486-a2ba-aa1d18a36c31 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.584395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 298/597] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:29.585647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.615884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.616946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.617452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.617960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.628154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:29.630732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.632892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:29.654533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:29.655691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:29.656775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:29.656775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:29.656775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:29.658958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 300/598] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:29.660982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.751688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f945ed-5842-4be9-8a39-4090f74b2478 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.801297 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.801913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.802189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.803079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.803729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:29.804022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.805417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.814083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b3de4-cd68-4ef0-8021-655ba1ff0936 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.821331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 299/599] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:29.822354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.847428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:29.866235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08293c0d-7c7e-4f8b-9a6a-89418ff91d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:29.866885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08293c0d-7c7e-4f8b-9a6a-89418ff91d30 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.867157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08293c0d-7c7e-4f8b-9a6a-89418ff91d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.867421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08293c0d-7c7e-4f8b-9a6a-89418ff91d30 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/4e395e059b5d436a8678bb302bcfafb5/roles/a9f22cef53e740cdbc09d1137362031e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.883139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08293c0d-7c7e-4f8b-9a6a-89418ff91d30 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=4e395e059b5d436a8678bb302bcfafb5, role_id=a9f22cef53e740cdbc09d1137362031e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:29.884576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08293c0d-7c7e-4f8b-9a6a-89418ff91d30 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:29.899609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 301/600] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:25:29 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/4e395e059b5d436a8678bb302bcfafb5/roles/a9f22cef53e740cdbc09d1137362031e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:29.899609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b81fdf-bd3f-413e-98e9-313f99e6156c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:29.899609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b81fdf-bd3f-413e-98e9-313f99e6156c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:29.899609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b81fdf-bd3f-413e-98e9-313f99e6156c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:29.899609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 302/601] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:29.899609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b81fdf-bd3f-413e-98e9-313f99e6156c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:29.957106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975da2b8-c415-4fc3-9da1-ab09c1e03141 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:29.981647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.001051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29aca-e991-4bf1-bb30-13def17227b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.002032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29aca-e991-4bf1-bb30-13def17227b1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.002765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29aca-e991-4bf1-bb30-13def17227b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.003144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29aca-e991-4bf1-bb30-13def17227b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.004842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e29aca-e991-4bf1-bb30-13def17227b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:30.006530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e29aca-e991-4bf1-bb30-13def17227b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.008765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e29aca-e991-4bf1-bb30-13def17227b1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.025421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 300/602] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:30.027688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1f01-612b-4f0b-bcdf-1f0448c58424 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.028077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1f01-612b-4f0b-bcdf-1f0448c58424 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.028650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1f01-612b-4f0b-bcdf-1f0448c58424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.054388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9d5b1f01-612b-4f0b-bcdf-1f0448c58424 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:30.055790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9d5b1f01-612b-4f0b-bcdf-1f0448c58424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:30.056704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b81fdf-bd3f-413e-98e9-313f99e6156c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.107894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.108439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.108716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.108981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.119589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.121079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:30.133899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:30.135006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:30.136135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:30.136135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:30.136135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:30.136135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:30.136135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:30.136135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:30.136135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:30.136135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:30.139817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 303/603] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:30.140730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.154994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 301/604] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:30.156093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b1f01-612b-4f0b-bcdf-1f0448c58424 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.170695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.276816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fb252-833f-4d9b-b7a3-7426eb7e6aa6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.280674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.290354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b1f01-612b-4f0b-bcdf-1f0448c58424 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.326291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.326710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.327159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.327323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.327604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.327705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.327876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.328134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.328569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:30.328872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.330376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.337022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.339160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.348087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 302/605] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:30.349013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:30.352491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:30.353616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:30.354686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 304/606] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:30.355262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.446668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.481660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9b4e3-bb1b-4d0f-83be-cb6df17a66a2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.481993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3766f-ee12-444d-bb99-55356979532c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.519426 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32adb87-cfab-4142-b684-13643349a51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.520168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32adb87-cfab-4142-b684-13643349a51b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.520372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32adb87-cfab-4142-b684-13643349a51b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.520601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32adb87-cfab-4142-b684-13643349a51b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.531910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32adb87-cfab-4142-b684-13643349a51b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:30.533844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca506f0-a36f-4498-9eb6-232f721ae8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.534834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca506f0-a36f-4498-9eb6-232f721ae8c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.535305 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca506f0-a36f-4498-9eb6-232f721ae8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.535768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca506f0-a36f-4498-9eb6-232f721ae8c2 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/b96be241231247b39e544529a4cd0b74/roles/a9f22cef53e740cdbc09d1137362031e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d32adb87-cfab-4142-b684-13643349a51b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:30.537107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:30.538142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:30.538964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:30.543427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 303/607] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:30.544327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32adb87-cfab-4142-b684-13643349a51b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.557007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca506f0-a36f-4498-9eb6-232f721ae8c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=b96be241231247b39e544529a4cd0b74, role_id=a9f22cef53e740cdbc09d1137362031e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.558993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca506f0-a36f-4498-9eb6-232f721ae8c2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.578572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 305/608] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:25:30 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/b96be241231247b39e544529a4cd0b74/roles/a9f22cef53e740cdbc09d1137362031e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:30.579875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca506f0-a36f-4498-9eb6-232f721ae8c2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.619456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.716929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32adb87-cfab-4142-b684-13643349a51b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.751908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:30.760403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9a5a3-7fc8-45fd-b499-492051e426ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.761212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9a5a3-7fc8-45fd-b499-492051e426ce None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.761674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9a5a3-7fc8-45fd-b499-492051e426ce None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.762065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9a5a3-7fc8-45fd-b499-492051e426ce None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.762934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9a5a3-7fc8-45fd-b499-492051e426ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:30.763349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9a5a3-7fc8-45fd-b499-492051e426ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.764555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca506f0-a36f-4498-9eb6-232f721ae8c2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.765019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9a5a3-7fc8-45fd-b499-492051e426ce None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.781038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 304/609] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:30.782361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fba58d-197c-44d7-a007-9bd557ceb2f4 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.782643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fba58d-197c-44d7-a007-9bd557ceb2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.782922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fba58d-197c-44d7-a007-9bd557ceb2f4 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.784465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 305/610] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:30.791141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fba58d-197c-44d7-a007-9bd557ceb2f4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.810497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f6898-6c9a-4b39-bef3-045ffef855bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.811488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f6898-6c9a-4b39-bef3-045ffef855bd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.811906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f6898-6c9a-4b39-bef3-045ffef855bd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.812329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f6898-6c9a-4b39-bef3-045ffef855bd None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.813144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f6898-6c9a-4b39-bef3-045ffef855bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:30.813469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f6898-6c9a-4b39-bef3-045ffef855bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:30.814873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f6898-6c9a-4b39-bef3-045ffef855bd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:30.830711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 306/611] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:30.832428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c7eab-6786-4b5b-b43d-29b37e27c305 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.832862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c7eab-6786-4b5b-b43d-29b37e27c305 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.833467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c7eab-6786-4b5b-b43d-29b37e27c305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.854370 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5f4c7eab-6786-4b5b-b43d-29b37e27c305 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:30.855988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5f4c7eab-6786-4b5b-b43d-29b37e27c305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:30.933712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fba58d-197c-44d7-a007-9bd557ceb2f4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:30.973520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 307/612] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:30.974365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c7eab-6786-4b5b-b43d-29b37e27c305 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:30.981410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d551a-7a13-4d16-972b-15c39ff34d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:30.982145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d551a-7a13-4d16-972b-15c39ff34d79 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:30.982911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d551a-7a13-4d16-972b-15c39ff34d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:30.983192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d551a-7a13-4d16-972b-15c39ff34d79 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/234eed198ab84e95ae1a7f03785a1481/roles/a9f22cef53e740cdbc09d1137362031e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:30.984296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:31.011391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d551a-7a13-4d16-972b-15c39ff34d79 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=234eed198ab84e95ae1a7f03785a1481, role_id=a9f22cef53e740cdbc09d1137362031e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.012832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d551a-7a13-4d16-972b-15c39ff34d79 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.037353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 306/613] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:25:30 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/234eed198ab84e95ae1a7f03785a1481/roles/a9f22cef53e740cdbc09d1137362031e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:25:31.037353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d551a-7a13-4d16-972b-15c39ff34d79 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.136054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c7eab-6786-4b5b-b43d-29b37e27c305 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.169750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:31.200778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.201459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.201734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.202002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.226472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.228757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:31.261765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:31.262804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:31.264070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:31.264070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:31.264070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:31.264070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 308/614] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:31.272497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.317621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d551a-7a13-4d16-972b-15c39ff34d79 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.354212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:31.363365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26f042-d0e7-4182-8f5a-221522a13c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.364282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26f042-d0e7-4182-8f5a-221522a13c53 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.364804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26f042-d0e7-4182-8f5a-221522a13c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.365287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26f042-d0e7-4182-8f5a-221522a13c53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.381802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26f042-d0e7-4182-8f5a-221522a13c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-8b26f042-d0e7-4182-8f5a-221522a13c53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:31.388755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:31.389991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:31.391056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:31.393506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 307/615] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:31.401253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26f042-d0e7-4182-8f5a-221522a13c53 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.415501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a56c19-86f8-42d9-b302-18ba55024329 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.453924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.454863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.455244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.455730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.456489 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:31.456888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.458391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.473138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 309/616] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:31.478938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.597590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48d3e0-4063-43e9-8ee9-ec48649a09d3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.617576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:31.639467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.640260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.640638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.641007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.649709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.651458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:31.664653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:31.665763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:31.666721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 310/617] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:31.673010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.717555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:31.722736 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26f042-d0e7-4182-8f5a-221522a13c53 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:31.816612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:31.817582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:31.818126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:31.818571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:31.819737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:31.820241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:31.822787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:31.852746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 308/618] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:31.858754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:31.969840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab0f9a-a6fb-4b5b-9848-2258e4d075e3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.054328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.055127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.055551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.055957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.056825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:32.057301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.060774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.080349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d4cce-3841-4401-ab3e-04013619a4f7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.101193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 311/619] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 10 13:25:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:32.106496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.150511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f537a74-04e3-4f28-b9e3-5383c0ce61e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.151572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537a74-04e3-4f28-b9e3-5383c0ce61e1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.152053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537a74-04e3-4f28-b9e3-5383c0ce61e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.152526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537a74-04e3-4f28-b9e3-5383c0ce61e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.173636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:32.178987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f537a74-04e3-4f28-b9e3-5383c0ce61e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.180498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f537a74-04e3-4f28-b9e3-5383c0ce61e1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7f537a74-04e3-4f28-b9e3-5383c0ce61e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:32.206319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:32.207337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:32.208241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:32.208241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:32.208241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:32.213441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 309/620] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:32.214860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54da0cf-8127-499e-a314-a204513d8bd9 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.215139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54da0cf-8127-499e-a314-a204513d8bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.215406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54da0cf-8127-499e-a314-a204513d8bd9 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.217043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 310/621] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:32.222341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54da0cf-8127-499e-a314-a204513d8bd9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.258003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:32.279560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3d67c-e070-4d2c-ac6e-8ced7bea8388 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.335251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b49371-c0b4-4f9f-8272-18ccef3a6701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.335825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b49371-c0b4-4f9f-8272-18ccef3a6701 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.336102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b49371-c0b4-4f9f-8272-18ccef3a6701 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.336364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b49371-c0b4-4f9f-8272-18ccef3a6701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.351639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b49371-c0b4-4f9f-8272-18ccef3a6701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.353112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b49371-c0b4-4f9f-8272-18ccef3a6701 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-31b49371-c0b4-4f9f-8272-18ccef3a6701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:32.376120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:32.377118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:32.378014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:32.378014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:32.378014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:32.378620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 312/622] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:32.383475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9afadb-ef8e-4e42-9aef-8e8e87ef5311 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.383475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9afadb-ef8e-4e42-9aef-8e8e87ef5311 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.383475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9afadb-ef8e-4e42-9aef-8e8e87ef5311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.393581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54da0cf-8127-499e-a314-a204513d8bd9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.400279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-eb9afadb-ef8e-4e42-9aef-8e8e87ef5311 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:32.401566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-eb9afadb-ef8e-4e42-9aef-8e8e87ef5311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:32.434628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.435256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.435533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.435862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.436519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:32.436817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.438214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.451184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 311/623] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:32.451983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.500166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 313/624] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:32.501160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9afadb-ef8e-4e42-9aef-8e8e87ef5311 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.523110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:32.567406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:32.599740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e11cf3-e06a-4cc7-af50-861c4373fd84 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.626151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9afadb-ef8e-4e42-9aef-8e8e87ef5311 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.638566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.639476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.639906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.640341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.641128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:32.641604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.643149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.656804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 312/625] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:32.658032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.668448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.669099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.669410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.669680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.678037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.679507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:32.691596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:32.692830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 13:25:32.694627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:32.698578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 314/626] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:32.699506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.774598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af002ed5-8c17-4c8f-8c72-8b6361a96e02 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.783675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:32.810413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e5570-2d7f-45b0-86fb-e93903a47168 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.812599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1034c-22dc-4a1e-b0fc-49380e9ec926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.813455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1034c-22dc-4a1e-b0fc-49380e9ec926 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.813906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1034c-22dc-4a1e-b0fc-49380e9ec926 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.814318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1034c-22dc-4a1e-b0fc-49380e9ec926 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.816509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:32.828134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1034c-22dc-4a1e-b0fc-49380e9ec926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-15d1034c-22dc-4a1e-b0fc-49380e9ec926 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:32.833796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:32.834919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:32.835822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:32.837966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 313/627] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:32.843551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1034c-22dc-4a1e-b0fc-49380e9ec926 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.847678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.848318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.848595 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.848868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.856874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:32.858533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:32.871099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:32.872282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:32.873179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 315/628] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:32.877261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:32.958003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1034c-22dc-4a1e-b0fc-49380e9ec926 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.984816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf1fd1-f8e8-40b0-8301-4399c39cd8fb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:32.996937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:32.997854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:32.998302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:32.998717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:32.999651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:33.000098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.001772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.021611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.021878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 314/629] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:33.022861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.026791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.027828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.027828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.028120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:33.028423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.030153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.032840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.046656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 316/630] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:33.046656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.099869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.166506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272778-ffa2-4e07-af06-4d639113b58a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.191209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603a6a-5854-4745-a865-d4b13e4e08dc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.225811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7def95-5fec-4b0a-8e0c-f75e5db6c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.226443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7def95-5fec-4b0a-8e0c-f75e5db6c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.226725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7def95-5fec-4b0a-8e0c-f75e5db6c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.226981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7def95-5fec-4b0a-8e0c-f75e5db6c8b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.230382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.231483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.231967 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.232446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.233982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:33.234519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.237407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.257665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 315/631] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:33.258510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.317374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.322048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7def95-5fec-4b0a-8e0c-f75e5db6c8b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.324063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7def95-5fec-4b0a-8e0c-f75e5db6c8b1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.364700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96cc7-ff55-4af9-b2f9-d635d7f1f65f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.378289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.399225 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a316d2f-2f43-4d35-8dea-87a29e7e19db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.399852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316d2f-2f43-4d35-8dea-87a29e7e19db None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.400126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316d2f-2f43-4d35-8dea-87a29e7e19db None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.400390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316d2f-2f43-4d35-8dea-87a29e7e19db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.412838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a316d2f-2f43-4d35-8dea-87a29e7e19db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.414335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a316d2f-2f43-4d35-8dea-87a29e7e19db None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5a316d2f-2f43-4d35-8dea-87a29e7e19db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:33.428981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:33.429976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:33.430839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:33.430839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:33.430839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:33.430839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 316/632] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:33.431292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31e53-9f7a-4b35-ae8b-f56b84ec9b11 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.431519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31e53-9f7a-4b35-ae8b-f56b84ec9b11 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.431742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31e53-9f7a-4b35-ae8b-f56b84ec9b11 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.433155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 317/633] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:33.433624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc31e53-9f7a-4b35-ae8b-f56b84ec9b11 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.439472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 317/634] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:33.441101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186ebf9-c823-45eb-8049-d91a5929f1b3 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.441418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186ebf9-c823-45eb-8049-d91a5929f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.441694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186ebf9-c823-45eb-8049-d91a5929f1b3 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.443494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 318/635] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:33.444275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186ebf9-c823-45eb-8049-d91a5929f1b3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.559022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.561331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc31e53-9f7a-4b35-ae8b-f56b84ec9b11 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.564894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186ebf9-c823-45eb-8049-d91a5929f1b3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.600059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805afc5-e232-48e3-a00b-448af77203a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.600833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805afc5-e232-48e3-a00b-448af77203a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.601105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805afc5-e232-48e3-a00b-448af77203a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.601431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805afc5-e232-48e3-a00b-448af77203a3 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/10209fea1bcc43509534b7e20f635c75/roles/a9f22cef53e740cdbc09d1137362031e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.601585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dff3db-f401-40d6-87d6-5fe098c96cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.602140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dff3db-f401-40d6-87d6-5fe098c96cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.602418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dff3db-f401-40d6-87d6-5fe098c96cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.602683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dff3db-f401-40d6-87d6-5fe098c96cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.603329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dff3db-f401-40d6-87d6-5fe098c96cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:33.603618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dff3db-f401-40d6-87d6-5fe098c96cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.605016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dff3db-f401-40d6-87d6-5fe098c96cf2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.626226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805afc5-e232-48e3-a00b-448af77203a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=10209fea1bcc43509534b7e20f635c75, role_id=a9f22cef53e740cdbc09d1137362031e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.626459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 319/636] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:33.626556 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805afc5-e232-48e3-a00b-448af77203a3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.626649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb3c9-236f-4fdc-b5b2-bfb11cf1a157 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.626649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb3c9-236f-4fdc-b5b2-bfb11cf1a157 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.626649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb3c9-236f-4fdc-b5b2-bfb11cf1a157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.636351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 318/637] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:25:33 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/10209fea1bcc43509534b7e20f635c75/roles/a9f22cef53e740cdbc09d1137362031e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:33.637170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805afc5-e232-48e3-a00b-448af77203a3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.639863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5a7fb3c9-236f-4fdc-b5b2-bfb11cf1a157 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:33.641092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5a7fb3c9-236f-4fdc-b5b2-bfb11cf1a157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:33.655182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.729589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 320/638] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:33.730792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdb781-e5f3-4924-b28e-92dadf3e1106 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.731067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdb781-e5f3-4924-b28e-92dadf3e1106 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.731327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdb781-e5f3-4924-b28e-92dadf3e1106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.759253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f1cdb781-e5f3-4924-b28e-92dadf3e1106 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:33.760721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f1cdb781-e5f3-4924-b28e-92dadf3e1106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:33.789493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805afc5-e232-48e3-a00b-448af77203a3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:33.834405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:33.835265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:33.841253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:33.841253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:33.850987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:33.852471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:33.861935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 321/639] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:33.863073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdb781-e5f3-4924-b28e-92dadf3e1106 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.884649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:33.892373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:33.893433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:33.894338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:33.894338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:33.894338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:33.894841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 319/640] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:33.897590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:33.976943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:33.981822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdb781-e5f3-4924-b28e-92dadf3e1106 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.016991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703d3a-59d1-4fb0-840b-e5d2f136cfc6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.019745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.020453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.020733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.021043 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.028743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.030223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:34.043181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:34.044172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 322/641] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:34.045104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.064094 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2335ea-e1b7-4736-b89e-b8df4e06ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.065431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2335ea-e1b7-4736-b89e-b8df4e06ef06 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.066178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2335ea-e1b7-4736-b89e-b8df4e06ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.066691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2335ea-e1b7-4736-b89e-b8df4e06ef06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.083278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2335ea-e1b7-4736-b89e-b8df4e06ef06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-fe2335ea-e1b7-4736-b89e-b8df4e06ef06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:34.090290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:34.091521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:34.092543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:34.094549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 320/642] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:34.095527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2335ea-e1b7-4736-b89e-b8df4e06ef06 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.154721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.178551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e59349-38f7-4705-bb08-dccb53e3df0c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.207049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2335ea-e1b7-4736-b89e-b8df4e06ef06 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.215447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.216323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.216749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.217289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.218070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:34.218520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.220268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.231800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.238183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 323/643] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:34.238903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.241896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9356d6-563d-47a5-980b-9ad4f108fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.242519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9356d6-563d-47a5-980b-9ad4f108fee8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.242792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9356d6-563d-47a5-980b-9ad4f108fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.243058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9356d6-563d-47a5-980b-9ad4f108fee8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.256615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9356d6-563d-47a5-980b-9ad4f108fee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ac9356d6-563d-47a5-980b-9ad4f108fee8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:34.262191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:34.263099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:34.263996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 321/644] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:34.265903 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9356d6-563d-47a5-980b-9ad4f108fee8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.347603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298d109-09bc-4a82-b37d-6fea188a8b6a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.377212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9356d6-563d-47a5-980b-9ad4f108fee8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.383056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.383657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.383933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.384203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.384929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:34.385242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.386685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.394633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.410188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 324/645] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:34.410188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.420225 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.420835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.421142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.421404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.422165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:34.422352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.423683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.436537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 322/646] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:34.436537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.487547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.519396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb51785-f5a2-4391-91f4-0c2d8cd8b060 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.542119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee318fa2-a80a-422b-b3df-936c98137a1f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.553880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.554503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.554777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.555064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.563000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.564447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:34.575843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:34.576899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 325/647] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:34.578054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.585645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.586268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.586544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.586812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.587609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:34.587910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.589298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.597656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 323/648] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:34.598435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.651500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.709406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99740-88fd-48b8-ba6f-bc53ddd28843 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.738093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d97ade-b993-47e4-be20-7e0e1131e3e2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.751268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.751458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8ceb0-a616-41ca-ad0f-34e1b6a40590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.752049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ceb0-a616-41ca-ad0f-34e1b6a40590 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.752551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ceb0-a616-41ca-ad0f-34e1b6a40590 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.752551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ceb0-a616-41ca-ad0f-34e1b6a40590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.785239 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:34.786182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:34.786786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:34.787060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:34.801179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.803314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:34.820187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:34.821197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:34.822124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:34.822124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:34.822124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:34.822124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 324/649] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:34.822346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:34.855801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8ceb0-a616-41ca-ad0f-34e1b6a40590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:34.858713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8ceb0-a616-41ca-ad0f-34e1b6a40590 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:34.945978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:34.957222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4268aa2-5b6b-4507-8da1-a755efdc83f3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:34.991747 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 326/650] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:34.992699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8ceb0-a616-41ca-ad0f-34e1b6a40590 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.002671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.004106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.004993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.006063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.007267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:35.007768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.010420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.030106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 325/651] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:35.030853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.069124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.104444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8ceb0-a616-41ca-ad0f-34e1b6a40590 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.141999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.143323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.143763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.144031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.151231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4cab-6007-44bd-b9c0-80b7be7e9682 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.156020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.157748 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:35.179697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:35.180739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:35.182329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:35.182329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:35.182329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:35.182329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 327/652] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:35.182329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.194559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.300698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316226f-76c0-4bc5-ac79-b914e9d57635 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.308937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.309580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.309857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.310121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.310759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:35.311061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.312422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.324181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 326/653] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:35.329265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.336512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.337136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.337432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.337698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.351216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.352682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:35.368221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:35.369283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:35.370187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:35.370187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:35.370187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:35.370187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 328/654] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:35.371201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.421503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.434026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.441946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c12012-ea0a-4f6d-b236-a46d42eeedd2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.478881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.479486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.479753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.479893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72651d0-f086-495e-b16f-94ffed116bd5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.480013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.480711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=ce851cf0509947e783dbd5b4809bc897) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:35.481007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.482587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.489004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 327/655] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/role_assignments?user.id=ce851cf0509947e783dbd5b4809bc897&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:35.489865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.531692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.533318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.534194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.534655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.535812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:35.536343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.539509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.556658 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 329/656] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:35.557560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.609730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b3d70-8008-46f9-a8a1-58b8a16ce9c1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.651338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.652338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.652818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.653300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.662402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.664812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:35.678962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:35.679973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:25:35.680896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:35.683341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 328/657] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:35.684356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.696095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.718721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163c61c-741f-451f-a009-b1b455dacc2c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.743498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.766703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e12d9e-e4a1-4758-a737-ebba49107693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.767902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e12d9e-e4a1-4758-a737-ebba49107693 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.768179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e12d9e-e4a1-4758-a737-ebba49107693 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.768439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e12d9e-e4a1-4758-a737-ebba49107693 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.769085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e12d9e-e4a1-4758-a737-ebba49107693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:35.769403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e12d9e-e4a1-4758-a737-ebba49107693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.770782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e12d9e-e4a1-4758-a737-ebba49107693 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.783797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 330/658] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:35.785040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e798525-1c24-481a-a574-f0bb00c1bb60 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.785380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e798525-1c24-481a-a574-f0bb00c1bb60 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.785684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e798525-1c24-481a-a574-f0bb00c1bb60 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.787126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 331/659] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:35.787832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e798525-1c24-481a-a574-f0bb00c1bb60 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.816034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75276d-60f3-4b55-979b-ff2524f3828c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.856686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3ab5c-67df-43d6-ab70-9c7bb3e99ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.857338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ab5c-67df-43d6-ab70-9c7bb3e99ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.857854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ab5c-67df-43d6-ab70-9c7bb3e99ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.858123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ab5c-67df-43d6-ab70-9c7bb3e99ea4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.867748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3ab5c-67df-43d6-ab70-9c7bb3e99ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.869255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3ab5c-67df-43d6-ab70-9c7bb3e99ea4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d1c3ab5c-67df-43d6-ab70-9c7bb3e99ea4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:35.881995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:35.883007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 13:25:35.883940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:35.887737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 329/660] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:35.888892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4926c3-5f74-47c2-83e5-9ad6a85dd928 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.889170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4926c3-5f74-47c2-83e5-9ad6a85dd928 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.889479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4926c3-5f74-47c2-83e5-9ad6a85dd928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.908433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8c4926c3-5f74-47c2-83e5-9ad6a85dd928 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:35.909749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8c4926c3-5f74-47c2-83e5-9ad6a85dd928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:35.917049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e798525-1c24-481a-a574-f0bb00c1bb60 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:35.955787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:35.956397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:35.957034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:35.957347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:35.958084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:35.959148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:35.960856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:35.975961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:35.988682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 332/661] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:35.988907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 330/662] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:35.988907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4926c3-5f74-47c2-83e5-9ad6a85dd928 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:35.997614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.043207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.112623 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4926c3-5f74-47c2-83e5-9ad6a85dd928 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.121314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac07f1-f6c5-47b5-8d7b-563436f0c94c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.156795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.157766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.158111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.158508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.159622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:36.160192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.162876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.183485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.183485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.183485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.183485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.189321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 331/663] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:36.194263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.194409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.194947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:36.208037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:36.209843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:36.210950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:36.210950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:36.210950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:36.210950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:36.210950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:36.210950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:36.210950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:36.210950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:36.216990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 333/664] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:36.216990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.268226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.323775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34ef0-463c-42e7-b2a4-f8571a4a9772 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.329377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.334201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436c212-40d7-4057-8449-c739f0b9cfca None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.363022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e7727-2fb1-419c-9dec-42acc699b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.363724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7727-2fb1-419c-9dec-42acc699b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.363998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7727-2fb1-419c-9dec-42acc699b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.364256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7727-2fb1-419c-9dec-42acc699b7e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.370135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66dde1-0fa4-4aa0-aebe-c3183e6f2fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.370757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66dde1-0fa4-4aa0-aebe-c3183e6f2fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.371034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66dde1-0fa4-4aa0-aebe-c3183e6f2fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.371300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66dde1-0fa4-4aa0-aebe-c3183e6f2fb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.381915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66dde1-0fa4-4aa0-aebe-c3183e6f2fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-dc66dde1-0fa4-4aa0-aebe-c3183e6f2fb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:36.386631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:36.387585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:36.388450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 334/665] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:36.389153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66dde1-0fa4-4aa0-aebe-c3183e6f2fb5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.453885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e7727-2fb1-419c-9dec-42acc699b7e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.455549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e7727-2fb1-419c-9dec-42acc699b7e1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.530218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.531643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66dde1-0fa4-4aa0-aebe-c3183e6f2fb5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.571661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdab51a-aa8c-4971-9461-86e291494d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.572306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdab51a-aa8c-4971-9461-86e291494d2f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.572588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdab51a-aa8c-4971-9461-86e291494d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.572853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdab51a-aa8c-4971-9461-86e291494d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.581409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 332/666] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:36.585600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e7727-2fb1-419c-9dec-42acc699b7e1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.602092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.647135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdab51a-aa8c-4971-9461-86e291494d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.648535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdab51a-aa8c-4971-9461-86e291494d2f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.698615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e7727-2fb1-419c-9dec-42acc699b7e1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.737018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.737653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.737930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.738196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.738876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:36.739175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.741161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.758882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 333/667] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:36.758882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.766584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 335/668] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:36.767506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdab51a-aa8c-4971-9461-86e291494d2f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.782032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.846293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:36.889994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35af9d2-3235-491d-8a70-7254e896a479 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.898763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdab51a-aa8c-4971-9461-86e291494d2f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:36.927125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.928382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.928657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.929029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.929630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:36.929921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.931352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.938671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:36.939295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:36.939566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:36.939835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:36.944727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 334/669] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:36.944727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:36.955016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:36.956675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:36.977135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:36.978211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:36.979096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:36.979096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:36.979096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:36.979096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 336/670] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:36.980033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.034699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.068396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9076f-294e-4472-bae8-88832350fce1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.092482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792794e3-8476-4650-b99b-747ef3b687e6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.105314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.117984 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.118604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.118879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.119148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.119875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=4e395e059b5d436a8678bb302bcfafb5) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:37.120181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.121644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.128616 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 335/671] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/role_assignments?user.id=4e395e059b5d436a8678bb302bcfafb5&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:37.129626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.147452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.149718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.150528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.151004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.169195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.171157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:37.193793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:37.194807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:37.195685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:37.195685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:37.195685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:37.200389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 337/672] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:37.201502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.279631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494ea62-e036-4cc1-b092-89b258f2960d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.320028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.320525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.320802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.321077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.330402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.333572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.335131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:37.354135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:37.355346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:37.356464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:37.356464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:37.356464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:37.357114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b587f7-5a80-4356-845c-bdddd68b082d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.359312 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 336/673] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:37.360377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.418715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.420265 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.420388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.420986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.422331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:37.422881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.425597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.449153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 338/674] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:37.450670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.498877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b092df-87af-440a-aaee-fc93dcce7bdf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.508831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.554316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.555669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.556128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.556570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.557956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:37.558622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.561300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.580840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 337/675] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:37.586226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.588434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8f10a-23ac-41a7-a0d6-c07a305cb3b1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.626557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.627191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.627466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.627738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.628485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:37.628713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.630120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.643391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 339/676] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:37.644286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.663155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.719563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07895693-0b5f-406b-989e-50e21d8a2954 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.740918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.762452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75210bda-fc59-41c2-b5a1-6bbb3a6c2638 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.763501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.764140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.764438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.764722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.773334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.774771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:37.790380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:37.792034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 338/677] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:37.793803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.812602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e5e4-75ea-409c-b59b-ed799c6ef934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.813844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e5e4-75ea-409c-b59b-ed799c6ef934 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.814590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e5e4-75ea-409c-b59b-ed799c6ef934 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.815045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e5e4-75ea-409c-b59b-ed799c6ef934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.825608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e5e4-75ea-409c-b59b-ed799c6ef934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.828047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e5e4-75ea-409c-b59b-ed799c6ef934 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8079e5e4-75ea-409c-b59b-ed799c6ef934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:37.842703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:37.843671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:37.844592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:37.844592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:37.844592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:37.844592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:37.844592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:37.844592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:37.844592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:37.844592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:37.845031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 340/678] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:37.846146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35ee5f-50ad-4cc3-818b-800fdb865090 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.846429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35ee5f-50ad-4cc3-818b-800fdb865090 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.846699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35ee5f-50ad-4cc3-818b-800fdb865090 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.848148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 341/679] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:37.852749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35ee5f-50ad-4cc3-818b-800fdb865090 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:37.928251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3024a8-54a6-4ebf-8c6a-23b3750d097b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.943470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:37.966144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:37.966734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:37.967010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:37.967368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:37.968834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35ee5f-50ad-4cc3-818b-800fdb865090 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:37.976311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:37.977838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:37.990549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:37.991529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 339/680] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:37.992523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.012738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661213d8-1a63-4f84-99c4-ea8f66a8ca03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.013671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661213d8-1a63-4f84-99c4-ea8f66a8ca03 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.014029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661213d8-1a63-4f84-99c4-ea8f66a8ca03 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.014302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661213d8-1a63-4f84-99c4-ea8f66a8ca03 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.014997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661213d8-1a63-4f84-99c4-ea8f66a8ca03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:38.015296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661213d8-1a63-4f84-99c4-ea8f66a8ca03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.016731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661213d8-1a63-4f84-99c4-ea8f66a8ca03 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.029740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 342/681] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:38.030970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f954d-ebb0-4eec-8155-53cdce8d1d65 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.031289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f954d-ebb0-4eec-8155-53cdce8d1d65 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.031512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f954d-ebb0-4eec-8155-53cdce8d1d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.038373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.050544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b72f954d-ebb0-4eec-8155-53cdce8d1d65 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:38.051734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b72f954d-ebb0-4eec-8155-53cdce8d1d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:38.107390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1192c5-5381-4021-895f-fc3fd2b1cb14 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.126448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 343/682] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:38.127611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f954d-ebb0-4eec-8155-53cdce8d1d65 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.152265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.153284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.153766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.154031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.154710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:38.155008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.156600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.168880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 340/683] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:38.170054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.203902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.243401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f954d-ebb0-4eec-8155-53cdce8d1d65 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.279842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96aff3-6f94-43e4-9646-a94f6f1bc5b3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.283318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.289311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.289915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.290188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.290451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.291177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:38.291475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.292903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.309993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 344/684] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 10 13:25:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:38.310630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.318716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.319440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.319712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.319968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.320694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=b96be241231247b39e544529a4cd0b74) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:38.320982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.322419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.337275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 341/685] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:38 2022] GET /identity/v3/role_assignments?user.id=b96be241231247b39e544529a4cd0b74&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:38.337275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.438968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee96db6-c657-44e2-b671-697ed57361b3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.478953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.480434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.481412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.481897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.482332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.486357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ee9f5-8d21-4cd5-bece-d0b198a2c341 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.495587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.497410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:38.516351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:38.517412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:38.518317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:38.518317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:38.518317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:38.523801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 345/686] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:38.523801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.544276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.544884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.545155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.545442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.546162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=234eed198ab84e95ae1a7f03785a1481) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:38.546467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.547882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.554301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 342/687] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:38 2022] GET /identity/v3/role_assignments?user.id=234eed198ab84e95ae1a7f03785a1481&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:38.554918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.579352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.666016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b5046-2405-4962-94cb-ac8e41e7072b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.725286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6c704-96f1-4b1f-ad73-a8f81cf6f912 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.752139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8f99-f276-40be-ad42-4f525e37cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.752676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8f99-f276-40be-ad42-4f525e37cbaf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.752951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8f99-f276-40be-ad42-4f525e37cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.753219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8f99-f276-40be-ad42-4f525e37cbaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.794734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.795504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.795776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.796349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.797276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:38.797587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.799108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.816422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 343/688] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:38.822014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.852703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.858077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8f99-f276-40be-ad42-4f525e37cbaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.859744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8f99-f276-40be-ad42-4f525e37cbaf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:38.905651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:38.939854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18ba90-b3ad-4512-92dd-c3f2a4c9e896 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:38.972788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 346/689] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:38.978911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8f99-f276-40be-ad42-4f525e37cbaf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:38.980635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ccfb7-3d7a-480e-8822-c152dfb17805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:38.981533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ccfb7-3d7a-480e-8822-c152dfb17805 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:38.981765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ccfb7-3d7a-480e-8822-c152dfb17805 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:38.982042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ccfb7-3d7a-480e-8822-c152dfb17805 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:38.983001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ccfb7-3d7a-480e-8822-c152dfb17805 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:38.989838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ccfb7-3d7a-480e-8822-c152dfb17805 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:39.024300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 344/690] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:25:38 2022] POST /identity/v3/users => generated 254 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:39.106684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8f99-f276-40be-ad42-4f525e37cbaf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:39.165816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86b32e-a3b9-40ed-af61-46774b15117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:39.166776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86b32e-a3b9-40ed-af61-46774b15117f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.167218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86b32e-a3b9-40ed-af61-46774b15117f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.167633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86b32e-a3b9-40ed-af61-46774b15117f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.191676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86b32e-a3b9-40ed-af61-46774b15117f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:39.194221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86b32e-a3b9-40ed-af61-46774b15117f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:39.196695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-da86b32e-a3b9-40ed-af61-46774b15117f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:39.216931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:39.218154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:39.219450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:39.219450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:39.219450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:39.221449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 347/691] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:39.228944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ccfb7-3d7a-480e-8822-c152dfb17805 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:39.357420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:39.389061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ccfb7-3d7a-480e-8822-c152dfb17805 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:39.401920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a3b46-9b73-425e-b102-d16fa3dc67e3 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.402460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a3b46-9b73-425e-b102-d16fa3dc67e3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.402460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a3b46-9b73-425e-b102-d16fa3dc67e3 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.403979 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 348/692] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:39.411728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e3dd7-1d5a-40d9-ab49-bd5a3ca77012 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.412155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e3dd7-1d5a-40d9-ab49-bd5a3ca77012 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.412571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e3dd7-1d5a-40d9-ab49-bd5a3ca77012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.430946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:39.431853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.432272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.432682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.433520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:39.433977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:39.435367 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:39.441863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-091e3dd7-1d5a-40d9-ab49-bd5a3ca77012 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:39.444680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-091e3dd7-1d5a-40d9-ab49-bd5a3ca77012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:39.453296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 345/693] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:39.457119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:39.554430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 349/694] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:39.581425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e3dd7-1d5a-40d9-ab49-bd5a3ca77012 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:39.632697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7470-5c35-49c9-8e6c-52f321f0ffcf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:39.674172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:39.674962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.675293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.675501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.679002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:39.685114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:39.687203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:39.703130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:39.704134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:39.705007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:39.705007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:39.705007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:39.705007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:39.705007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:39.705007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:39.705007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:39.705007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:39.707680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 346/695] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:39.718339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:39.777936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e3dd7-1d5a-40d9-ab49-bd5a3ca77012 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:39.825003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:39.847746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8124e9-dd67-407e-8fce-031f3c69e8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:39.848384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8124e9-dd67-407e-8fce-031f3c69e8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.848657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8124e9-dd67-407e-8fce-031f3c69e8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.849469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8124e9-dd67-407e-8fce-031f3c69e8c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.869959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8124e9-dd67-407e-8fce-031f3c69e8c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bc8124e9-dd67-407e-8fce-031f3c69e8c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:39.877208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:39.878187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:39.879005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:39.880225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 350/696] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:39.887531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8124e9-dd67-407e-8fce-031f3c69e8c4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:39.905472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7869dd-848f-41cb-b9eb-7d8d83379b78 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:39.944777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:39.946051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:39.946326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:39.946769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:39.947517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:39.947823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:39.949263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:39.965373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 347/697] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:39.971527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:40.085787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:40.125150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8124e9-dd67-407e-8fce-031f3c69e8c4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:40.173948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:40.175082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:40.175555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:40.176011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:40.178538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:40.179873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:40.181497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:40.188067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d365f-d6ea-4dba-8796-b85e02a67f4f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:40.207001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 351/698] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:40.214306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:40.238845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:40.240080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:40.240309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:40.240583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:40.241351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=10209fea1bcc43509534b7e20f635c75) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:40.241669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:40.243175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:40.250170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 348/699] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:39 2022] GET /identity/v3/role_assignments?user.id=10209fea1bcc43509534b7e20f635c75&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:40.284576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:40.356330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eb1e7-5c71-4e6e-ac35-7f0504e70a96 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:40.404974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:40.406559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:40.407045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:40.407514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:40.426283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:40.428789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:40.453768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:40.454746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:40.455661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:40.455661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:40.455661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:40.457964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 352/700] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:40.463351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:40.624490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4eab6-4876-4abe-880d-18d76d56c0b6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:40.647871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:40.669080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be2733-fce5-4e10-b07d-f9837c31f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:40.669754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be2733-fce5-4e10-b07d-f9837c31f006 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:40.670085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be2733-fce5-4e10-b07d-f9837c31f006 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:40.670360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be2733-fce5-4e10-b07d-f9837c31f006 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:40.671082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be2733-fce5-4e10-b07d-f9837c31f006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:40.671396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be2733-fce5-4e10-b07d-f9837c31f006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:40.672801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be2733-fce5-4e10-b07d-f9837c31f006 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:40.692084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 349/701] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:40.697997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:40.779957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc1c3-9bd6-4032-9558-878ff51791af None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:40.780155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc1c3-9bd6-4032-9558-878ff51791af None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:40.780410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc1c3-9bd6-4032-9558-878ff51791af None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:40.782011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 350/702] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:40.787640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd867cc-6b14-4d98-ba86-215d63ec9f26 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:40.787924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd867cc-6b14-4d98-ba86-215d63ec9f26 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:40.788184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd867cc-6b14-4d98-ba86-215d63ec9f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:40.835097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ddd867cc-6b14-4d98-ba86-215d63ec9f26 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:40.835917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:40.836303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ddd867cc-6b14-4d98-ba86-215d63ec9f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:40.840374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e412b02-f4b4-4787-8ba1-36979866a8f6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:40.883615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:40.884601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:40.885077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:40.885580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:40.896700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:40.899726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:40.920148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:40.921149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:40.922107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:40.933219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 353/703] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:25:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:40.935433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:40.985411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 351/704] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:41.018991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd867cc-6b14-4d98-ba86-215d63ec9f26 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:41.075853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8d660-4d14-44c7-ae3d-4a6affeef482 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:41.129344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:41.129967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.130311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.130579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.131346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:41.131650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:41.133173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:41.154448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 354/705] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:25:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:41.160102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:41.180943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:41.284218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326213c9-52f9-4c30-afda-2d86ab24caa6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:41.315409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd867cc-6b14-4d98-ba86-215d63ec9f26 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:41.322358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b10d63-2a6b-4955-9084-4eec1239ab60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:41.323258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b10d63-2a6b-4955-9084-4eec1239ab60 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.323886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b10d63-2a6b-4955-9084-4eec1239ab60 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.324377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b10d63-2a6b-4955-9084-4eec1239ab60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.347237 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:41.405942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b10d63-2a6b-4955-9084-4eec1239ab60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:41.408464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b10d63-2a6b-4955-9084-4eec1239ab60 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:41.418449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:41.418797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.418992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.419243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.419834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:41.421218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:41.431229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:41.445145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:41.446279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:41.447133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:41.453902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 352/706] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 10 13:25:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:41.456149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:41.540292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 355/707] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 13:25:41.542030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43155c1d-644c-470e-988f-cee2f9a4d113 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.542520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43155c1d-644c-470e-988f-cee2f9a4d113 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.543142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43155c1d-644c-470e-988f-cee2f9a4d113 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.544880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 356/708] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:41.546266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43155c1d-644c-470e-988f-cee2f9a4d113 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:41.605577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cdf615-98fa-4a06-88ed-87fa4e9abee9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:41.653114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa5183-d440-49db-9036-7f4443d80f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:41.654631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa5183-d440-49db-9036-7f4443d80f35 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.655311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa5183-d440-49db-9036-7f4443d80f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.655744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa5183-d440-49db-9036-7f4443d80f35 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.656876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa5183-d440-49db-9036-7f4443d80f35 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:41.657395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa5183-d440-49db-9036-7f4443d80f35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:41.659810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa5183-d440-49db-9036-7f4443d80f35 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:41.667527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 353/709] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 10 13:25:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:41.668995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c943b-8684-4233-b908-6900e531816d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.669454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c943b-8684-4233-b908-6900e531816d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.669895 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c943b-8684-4233-b908-6900e531816d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.687651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43155c1d-644c-470e-988f-cee2f9a4d113 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:41.692238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d77c943b-8684-4233-b908-6900e531816d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:41.693909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d77c943b-8684-4233-b908-6900e531816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:41.712870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:41.728783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:41.742218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:41.743107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.744155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.744583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.760378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:41.763179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:41.775525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 354/710] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:41.776787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c943b-8684-4233-b908-6900e531816d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:41.784250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:41.785480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:41.786661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:41.786661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:41.786661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:41.789167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 357/711] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:41.802829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:41.892868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c943b-8684-4233-b908-6900e531816d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:41.929036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737754c6-77a7-4ad9-8a1f-bff578761fe4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:41.930579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac521d-a6e0-4c98-affb-5834ba6bb08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:41.931188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac521d-a6e0-4c98-affb-5834ba6bb08b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.931475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac521d-a6e0-4c98-affb-5834ba6bb08b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.931743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac521d-a6e0-4c98-affb-5834ba6bb08b None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.932377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac521d-a6e0-4c98-affb-5834ba6bb08b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:41.932677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac521d-a6e0-4c98-affb-5834ba6bb08b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:41.934078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac521d-a6e0-4c98-affb-5834ba6bb08b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:41.940482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 355/712] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 10 13:25:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:41.941725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103f6e-77a6-4c82-86d2-c375b7d91c83 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.942011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103f6e-77a6-4c82-86d2-c375b7d91c83 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.942280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103f6e-77a6-4c82-86d2-c375b7d91c83 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.943874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 356/713] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:41.944499 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80103f6e-77a6-4c82-86d2-c375b7d91c83 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:41.985583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b23a00-c664-4a0f-a35c-04ed7bef0c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:41.986328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b23a00-c664-4a0f-a35c-04ed7bef0c13 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:41.986950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b23a00-c664-4a0f-a35c-04ed7bef0c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:41.987298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b23a00-c664-4a0f-a35c-04ed7bef0c13 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:41.988616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b23a00-c664-4a0f-a35c-04ed7bef0c13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:41.989344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b23a00-c664-4a0f-a35c-04ed7bef0c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:41.991772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b23a00-c664-4a0f-a35c-04ed7bef0c13 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:41.992712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:42.013610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 358/714] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:42.015135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af164b2-14db-43fd-9152-d291e95a7018 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.015502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af164b2-14db-43fd-9152-d291e95a7018 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.015850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af164b2-14db-43fd-9152-d291e95a7018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.035815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3af164b2-14db-43fd-9152-d291e95a7018 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:42.037807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3af164b2-14db-43fd-9152-d291e95a7018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:42.065203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:42.085865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80103f6e-77a6-4c82-86d2-c375b7d91c83 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.119146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 359/715] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:42.119805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af164b2-14db-43fd-9152-d291e95a7018 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:42.129032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:42.129725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.130003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.130908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.131506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:42.132900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:42.144610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:42.146510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:42.147373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 357/716] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:41 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:42.152908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:42.268059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af164b2-14db-43fd-9152-d291e95a7018 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.278722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:42.288517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9d75d-1995-4385-8096-8d7e410049c6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.312094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:42.312995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.313444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.313885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.322376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:42.324042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:42.334397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:42.335020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.335296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.335571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.336222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] RBAC: Adding query filter params (name=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:42.336527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:42.337976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:42.338644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:42.339692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:42.340735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:42.340735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:42.340735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:42.340735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:42.340735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:42.340735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:42.340735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:42.340735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:42.342898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 360/717] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:42.347915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 358/718] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:25:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:42.348100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:42.348715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:42.383399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:42.474346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075763d-ef42-41bf-8544-0b63f97f51eb None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.495054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2362e-de78-405a-88a7-a2bdeefd309e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.525677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:42.526477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.526774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.527043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.527757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:42.528572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:42.530373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:42.537853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15f4ab-0b0c-41ef-984f-5a56ee2c6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:42.539011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15f4ab-0b0c-41ef-984f-5a56ee2c6a84 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.539514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15f4ab-0b0c-41ef-984f-5a56ee2c6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.540028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15f4ab-0b0c-41ef-984f-5a56ee2c6a84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.547192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 359/719] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:42.548424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:42.552505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15f4ab-0b0c-41ef-984f-5a56ee2c6a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5e15f4ab-0b0c-41ef-984f-5a56ee2c6a84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:42.559189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:42.560102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:42.560939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:42.563584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 361/720] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:42.564619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15f4ab-0b0c-41ef-984f-5a56ee2c6a84 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:42.582235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:42.683080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efd32e-97d7-4ace-a661-f7e7bcafd877 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.729181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:42.729834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.730110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.730380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.731013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] RBAC: Adding query filter params (type=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:42.731310 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:42.733928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:42.742754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 360/721] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:25:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:42.744424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:42.845465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:42.899904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15f4ab-0b0c-41ef-984f-5a56ee2c6a84 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.944021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:42.948578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08797c-cddd-485c-b401-c1d45a586b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:42.950431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08797c-cddd-485c-b401-c1d45a586b09 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.950730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08797c-cddd-485c-b401-c1d45a586b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.951004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08797c-cddd-485c-b401-c1d45a586b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.951956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08797c-cddd-485c-b401-c1d45a586b09 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=4e395e059b5d436a8678bb302bcfafb5) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:42.952270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08797c-cddd-485c-b401-c1d45a586b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:42.953766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08797c-cddd-485c-b401-c1d45a586b09 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:42.962444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 362/722] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:42 2022] GET /identity/v3/role_assignments?user.id=4e395e059b5d436a8678bb302bcfafb5&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 259 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:42.968315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03080ad-5824-44ea-a0f4-6adb66577fcc None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:42.971098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f0507-178b-4ed1-ba13-abcf352e9060 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.971378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f0507-178b-4ed1-ba13-abcf352e9060 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.971650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f0507-178b-4ed1-ba13-abcf352e9060 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:42.973152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 363/723] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:42.979028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79833e1f-23aa-47cc-a087-ef15674ec9fd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:42.979687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79833e1f-23aa-47cc-a087-ef15674ec9fd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:42.980176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79833e1f-23aa-47cc-a087-ef15674ec9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.000016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-79833e1f-23aa-47cc-a087-ef15674ec9fd None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:43.002212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-79833e1f-23aa-47cc-a087-ef15674ec9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:43.014755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:43.015740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:43.016029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:43.016304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.017321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:43.017637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:43.019095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:43.031699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 361/724] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:43.037565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:43.089667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 364/725] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 13:25:43.115123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79833e1f-23aa-47cc-a087-ef15674ec9fd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:43.189546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e21d66-9761-4430-bc6f-71c3bde4404d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:43.201614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:43.251036 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:43.251755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:43.252041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:43.252309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.265270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:43.266829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:43.274663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79833e1f-23aa-47cc-a087-ef15674ec9fd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:43.276591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:43.286845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:43.287902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:43.288778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:43.288778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:43.288778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:43.288778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 362/726] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:43.294309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:43.358892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3df0a5-f8e8-4c8e-bbf7-cbc0f09c85d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:43.359548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3df0a5-f8e8-4c8e-bbf7-cbc0f09c85d2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:43.359827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3df0a5-f8e8-4c8e-bbf7-cbc0f09c85d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:43.360102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3df0a5-f8e8-4c8e-bbf7-cbc0f09c85d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.390376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3df0a5-f8e8-4c8e-bbf7-cbc0f09c85d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-cf3df0a5-f8e8-4c8e-bbf7-cbc0f09c85d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:43.395876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:43.397073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:43.397963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 365/727] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:43.405763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3df0a5-f8e8-4c8e-bbf7-cbc0f09c85d2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:43.413711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fa537-e070-48ed-b88f-a13223cb27f3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:43.493930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:43.494551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:43.494827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:43.495098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.495814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:43.496120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:43.499515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:43.528506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 363/728] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:43.537645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:43.555091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3df0a5-f8e8-4c8e-bbf7-cbc0f09c85d2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:43.614455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:43.621586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:43.623986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:43.624989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:43.626784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.627668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:43.628144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:43.631143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:43.650391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 366/729] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:43.660548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:43.660763 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:43.727907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7835908-8202-47a5-8045-60be67817af8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:43.770011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:43.770845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:43.771112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:43.771375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.786154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:43.787687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:43.807360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:43.808372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:25:43.809323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:43.809853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 364/730] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:25:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:43.817099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:43.883037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c278ac-d931-491e-9286-add85bc9cb0c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:43.952945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:43.953532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:43.953811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:43.954081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:43.977085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:43.978620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:44.006164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:44.007187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:44.008071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:44.008071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:44.008071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:44.008071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 367/731] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:44.017351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:44.027311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d0c2d-f664-4477-bbdd-41939dcc9b03 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:44.049995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:44.064951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:44.065606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.065880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.066145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.067108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:44.067230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:44.069125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:44.089551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 365/732] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 10 13:25:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:44.102468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:44.113966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:44.199745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ba548-3488-4b94-8745-20c7471a9ce0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:44.253416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea2f03-f7cf-434c-a840-7e7e36777512 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:44.297590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db94a0-89c4-4a5a-aa18-a7f52db8ec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:44.297946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:44.298559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.298839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.299109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.299404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db94a0-89c4-4a5a-aa18-a7f52db8ec8d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.299763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:44.300062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:44.300304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db94a0-89c4-4a5a-aa18-a7f52db8ec8d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.300771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db94a0-89c4-4a5a-aa18-a7f52db8ec8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.301461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:44.323656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 368/733] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:44.330196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:44.373618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:44.375935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db94a0-89c4-4a5a-aa18-a7f52db8ec8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:44.377867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db94a0-89c4-4a5a-aa18-a7f52db8ec8d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:44.485149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 366/734] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:44.492201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db94a0-89c4-4a5a-aa18-a7f52db8ec8d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:44.513174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520abbde-b20e-478e-bdbe-34cf59607326 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:44.520810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:44.550530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a596a4-63e5-4af5-bb36-5dca0fdaf29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:44.551167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a596a4-63e5-4af5-bb36-5dca0fdaf29d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.551442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a596a4-63e5-4af5-bb36-5dca0fdaf29d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.551716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a596a4-63e5-4af5-bb36-5dca0fdaf29d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.560147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a596a4-63e5-4af5-bb36-5dca0fdaf29d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:44.561742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a596a4-63e5-4af5-bb36-5dca0fdaf29d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-15a596a4-63e5-4af5-bb36-5dca0fdaf29d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:44.574674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:44.575668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:44.576613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 369/735] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:44.577725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dc699-1159-455b-97d1-cff867bca5f9 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.577725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dc699-1159-455b-97d1-cff867bca5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.577882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dc699-1159-455b-97d1-cff867bca5f9 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.579284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 370/736] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:44.584210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705dc699-1159-455b-97d1-cff867bca5f9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:44.585722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:44.612118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db94a0-89c4-4a5a-aa18-a7f52db8ec8d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:44.652134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4b1b1-8d15-4f91-8f29-65e2fab99dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:44.653126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4b1b1-8d15-4f91-8f29-65e2fab99dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.653526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4b1b1-8d15-4f91-8f29-65e2fab99dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.653963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4b1b1-8d15-4f91-8f29-65e2fab99dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.665748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4b1b1-8d15-4f91-8f29-65e2fab99dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:44.667414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4b1b1-8d15-4f91-8f29-65e2fab99dd1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-c6e4b1b1-8d15-4f91-8f29-65e2fab99dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:44.684421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:44.685694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:44.686636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:44.686636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:44.686636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:44.691936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 367/737] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:44.694014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d421-7e88-48d8-96ea-fa8fd2310c87 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.694362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d421-7e88-48d8-96ea-fa8fd2310c87 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.694701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d421-7e88-48d8-96ea-fa8fd2310c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.702939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705dc699-1159-455b-97d1-cff867bca5f9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:44.714739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2af2d421-7e88-48d8-96ea-fa8fd2310c87 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:44.716135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2af2d421-7e88-48d8-96ea-fa8fd2310c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:44.740651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:44.741645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.742100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.742547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.743596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:44.744082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:44.745819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:44.766450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 371/738] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 10 13:25:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:44.767952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:44.796175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:44.806110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 368/739] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:44.807240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2d421-7e88-48d8-96ea-fa8fd2310c87 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:44.879364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:44.905948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13bb67-696c-4fb8-9c74-3ec7784ff825 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:44.930044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2d421-7e88-48d8-96ea-fa8fd2310c87 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:44.944802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:44.945693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.946071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.946439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:44.947180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:44.947576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:44.949119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:44.966065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 372/740] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:44.966065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:44.981438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa62b0-140b-4727-92a7-97ba4b6723fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:44.982311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa62b0-140b-4727-92a7-97ba4b6723fb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:44.983056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa62b0-140b-4727-92a7-97ba4b6723fb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:44.983569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa62b0-140b-4727-92a7-97ba4b6723fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.054878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:45.062604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa62b0-140b-4727-92a7-97ba4b6723fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:45.064291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa62b0-140b-4727-92a7-97ba4b6723fb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:45.072721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1174dcb-6b3f-4157-8303-cb079ada6b79 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:45.107449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936974a3-78b5-4112-a468-6774cb3a9a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:45.108408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936974a3-78b5-4112-a468-6774cb3a9a6c None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.108761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936974a3-78b5-4112-a468-6774cb3a9a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.109103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936974a3-78b5-4112-a468-6774cb3a9a6c None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.110093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936974a3-78b5-4112-a468-6774cb3a9a6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:45.111275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936974a3-78b5-4112-a468-6774cb3a9a6c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:45.134494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:45.141330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 373/741] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:25:44 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:45.142363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936974a3-78b5-4112-a468-6774cb3a9a6c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.189691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 369/742] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 13:25:45.191247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37200f0-d62e-47ed-8204-87eac83d8b20 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.191533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37200f0-d62e-47ed-8204-87eac83d8b20 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.191917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37200f0-d62e-47ed-8204-87eac83d8b20 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.193945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 370/743] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:45.194798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37200f0-d62e-47ed-8204-87eac83d8b20 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.266096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936974a3-78b5-4112-a468-6774cb3a9a6c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:45.309325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4343-22b8-42a0-9825-6adc254e6691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:45.310190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4343-22b8-42a0-9825-6adc254e6691 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.310465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4343-22b8-42a0-9825-6adc254e6691 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.310728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4343-22b8-42a0-9825-6adc254e6691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.318540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff4343-22b8-42a0-9825-6adc254e6691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:45.320051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff4343-22b8-42a0-9825-6adc254e6691 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:45.323246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:45.324182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37200f0-d62e-47ed-8204-87eac83d8b20 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-24ff4343-22b8-42a0-9825-6adc254e6691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:45.332659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:45.334105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:45.334987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:45.334987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:45.334987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:45.334987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:45.334987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:45.334987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:45.334987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:45.334987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:45.335416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 374/744] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:45.336937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b736f7e-4894-47c9-a63f-fc1c13a9e7e0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.336937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b736f7e-4894-47c9-a63f-fc1c13a9e7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.337123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b736f7e-4894-47c9-a63f-fc1c13a9e7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.355305 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0b736f7e-4894-47c9-a63f-fc1c13a9e7e0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:45.356975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0b736f7e-4894-47c9-a63f-fc1c13a9e7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:45.364745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8038b-91a7-45ad-8c48-83ea0d2e0092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:45.365496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8038b-91a7-45ad-8c48-83ea0d2e0092 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.365838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8038b-91a7-45ad-8c48-83ea0d2e0092 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.366200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8038b-91a7-45ad-8c48-83ea0d2e0092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.376635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8038b-91a7-45ad-8c48-83ea0d2e0092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:45.378115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8038b-91a7-45ad-8c48-83ea0d2e0092 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d6e8038b-91a7-45ad-8c48-83ea0d2e0092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:45.392699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:45.393862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:45.395059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:45.395059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:45.395059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:45.396701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 371/745] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:45.398562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f54a91-5759-4789-8de3-687d9ef7987b None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.398843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f54a91-5759-4789-8de3-687d9ef7987b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.399114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f54a91-5759-4789-8de3-687d9ef7987b None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.400507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 372/746] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:45.401977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f54a91-5759-4789-8de3-687d9ef7987b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.419468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:45.434073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 375/747] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:45.435423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5028-e437-4b70-85c4-b16bcbb0ab3c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.435795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5028-e437-4b70-85c4-b16bcbb0ab3c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.436156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee5028-e437-4b70-85c4-b16bcbb0ab3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.455638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b7ee5028-e437-4b70-85c4-b16bcbb0ab3c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:45.457002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b7ee5028-e437-4b70-85c4-b16bcbb0ab3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:45.525052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f54a91-5759-4789-8de3-687d9ef7987b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:45.526880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 376/748] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 13:25:45.527704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee5028-e437-4b70-85c4-b16bcbb0ab3c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.569066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:45.569623 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.569901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.570168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.570856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:45.571158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:45.572576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:45.588908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 373/749] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:45.589718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.597579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:45.645052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee5028-e437-4b70-85c4-b16bcbb0ab3c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:45.663947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:45.681108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:45.681735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.682011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.682281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.682914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:45.683212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:45.684582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:45.697376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 377/750] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:45.698231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.706352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be99da-217a-4051-9df9-b6091677cb0d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:45.742976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56599ad-2ebd-42cb-8739-fdc917ba9fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:45.743655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56599ad-2ebd-42cb-8739-fdc917ba9fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.743926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56599ad-2ebd-42cb-8739-fdc917ba9fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.744185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56599ad-2ebd-42cb-8739-fdc917ba9fb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.754221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56599ad-2ebd-42cb-8739-fdc917ba9fb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d56599ad-2ebd-42cb-8739-fdc917ba9fb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:45.759949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:45.760898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 374/751] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:45.761795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56599ad-2ebd-42cb-8739-fdc917ba9fb9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.809494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10b427-bcee-4818-ab5f-9eefe12502b5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:45.829848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:45.845819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2606eb-dc70-44d9-b242-5c6722da54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:45.846196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2606eb-dc70-44d9-b242-5c6722da54df None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.846469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2606eb-dc70-44d9-b242-5c6722da54df None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.846800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2606eb-dc70-44d9-b242-5c6722da54df None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.847437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2606eb-dc70-44d9-b242-5c6722da54df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:45.848859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2606eb-dc70-44d9-b242-5c6722da54df None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:45.864287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 378/752] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:25:45 2022] POST /identity/v3/services => generated 240 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:45.865119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2606eb-dc70-44d9-b242-5c6722da54df None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.883833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56599ad-2ebd-42cb-8739-fdc917ba9fb9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:45.932519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:45.961010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:45.962086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:45.962560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:45.963019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:45.964604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=1e0811556a0542b685c98dabbaed5b81) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:45.965139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:45.967750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:45.977220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 375/753] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:45 2022] GET /identity/v3/role_assignments?user.id=1e0811556a0542b685c98dabbaed5b81&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:45.978473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:45.995191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2606eb-dc70-44d9-b242-5c6722da54df None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:46.036016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:46.037298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.037644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.037914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.056261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:46.058700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:46.074246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:46.075258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:46.076156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:46.076156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:46.076156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:46.076156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:46.076156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:46.076156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:46.076156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:46.076156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:46.076583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 379/754] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:46.082576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:46.134585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9d57c-377f-4e65-b372-3ff82f51d229 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:46.145668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:46.192640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:46.193324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.193609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.193879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.194840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:46.195145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:46.196515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:46.206703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 376/755] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:46.213137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:46.255409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517dbe8-3df2-4b1f-b0b6-ef044bdf5c62 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:46.311219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:46.320203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:46.320811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.321087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.321382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.322080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:46.322376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:46.325029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:46.353648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 380/756] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:46.357570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:46.409119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccc5f9-83c8-49e0-a99d-48028082d05b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:46.490356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:46.494010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f98860-5893-4b49-8a71-64d80feef0df None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:46.494556 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:46.495233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.495516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.495789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.519708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:46.521775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:46.530322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:46.530974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.531255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.531522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.532245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=10209fea1bcc43509534b7e20f635c75) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:46.532540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:46.534048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:46.540590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 381/757] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:46 2022] GET /identity/v3/role_assignments?user.id=10209fea1bcc43509534b7e20f635c75&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:46.540730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:46.541994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:46.542859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:46.542859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:46.542859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:46.542859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 377/758] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:46.547746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:46.656888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13758e1a-3c8f-408a-a160-83419ddfd911 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:46.693567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:46.694189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.694465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.694734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.695394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:46.695688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:46.697259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:46.711062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 382/759] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:46.716739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:46.847116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3647cd-85f4-4961-835f-b74c26cd315a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:46.849770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:46.885726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:46.886356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:46.886729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:46.887000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:46.894778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:46.896241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:46.907494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:46.909573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:46.911755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 378/760] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:25:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:46.918535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:46.970339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:47.064294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11ed2-53c8-4714-9841-4f158b95d5f8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:47.111224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:47.111829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:47.112106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:47.112373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:47.113095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:47.113415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:47.114901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:47.140700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 383/761] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:25:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:47.146939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:47.312286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35487af1-f178-41d6-8ad0-24581819c4bd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:47.360890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe857fbb-07ad-451a-bad7-ceb2a4461896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:47.361926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe857fbb-07ad-451a-bad7-ceb2a4461896 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:47.362427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe857fbb-07ad-451a-bad7-ceb2a4461896 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:47.362910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe857fbb-07ad-451a-bad7-ceb2a4461896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:47.374672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe857fbb-07ad-451a-bad7-ceb2a4461896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:47.376396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe857fbb-07ad-451a-bad7-ceb2a4461896 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-fe857fbb-07ad-451a-bad7-ceb2a4461896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:47.390772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:47.391743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:47.392619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:47.392619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:47.392619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:47.394707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 379/762] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:47.402390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:47.518992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6cc74-3d4e-45b3-a0f2-b4c6504b0a4e None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:47.519613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6cc74-3d4e-45b3-a0f2-b4c6504b0a4e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:47.520083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6cc74-3d4e-45b3-a0f2-b4c6504b0a4e None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:47.522574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 380/763] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:47.528418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f60b6c-41c9-459e-958c-bccd392b8a7d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:47.529506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f60b6c-41c9-459e-958c-bccd392b8a7d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:47.530141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f60b6c-41c9-459e-958c-bccd392b8a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:47.562393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d3f60b6c-41c9-459e-958c-bccd392b8a7d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:47.563703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d3f60b6c-41c9-459e-958c-bccd392b8a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:47.566124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:47.589632 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:47.599435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c5333-f9d0-42d3-a313-7b9149510354 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:47.650682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 381/764] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:47.683877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f60b6c-41c9-459e-958c-bccd392b8a7d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:47.684359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:47.685531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:47.685953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:47.686366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:47.687149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:47.687580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:47.689095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:47.707784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 384/765] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:47.712511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:47.854116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f60b6c-41c9-459e-958c-bccd392b8a7d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:47.872531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bbd70-db49-45c7-b6ea-08ef6ebb9d4f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:47.913052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:47.914509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:47.915294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:47.916268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:47.917057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:47.918489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:47.919139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:47.920763 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:47.921575 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:47.924171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:47.929261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:47.932124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:47.941836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:47.943360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:47.944178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:47.950828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 382/766] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 10 13:25:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:47.955001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:47.957760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:47.958738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:47.959617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:47.959617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:47.959617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:47.959617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:47.959617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:47.959617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:47.959617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:47.959617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:47.964026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 385/767] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:47.971492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:47.980643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:47.980793 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:48.122161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221b368-ac48-4c0d-a8c7-d5b483bc5aac None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:48.156372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5beee-0af0-4155-ba9b-27e1f81529bb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:48.189626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:48.190242 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.190520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.190790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.191417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:48.191718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:48.194772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:48.207650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 383/768] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 10 13:25:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:48.214450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:48.214673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:48.215384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.215764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.216138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.217206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:48.217620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:48.219061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:48.233606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 386/769] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:48.240591 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:48.357604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:48.360608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:48.390791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4da2-536d-4577-8273-bad50eeed25d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:48.403523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa2cca-eecc-4138-bac5-8c8459608ec5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:48.427086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ee8d-d70b-4806-a6f8-919f9a0a78b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:48.427850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ee8d-d70b-4806-a6f8-919f9a0a78b5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.428126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ee8d-d70b-4806-a6f8-919f9a0a78b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.428720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ee8d-d70b-4806-a6f8-919f9a0a78b5 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.429644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33ee8d-d70b-4806-a6f8-919f9a0a78b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:48.429947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33ee8d-d70b-4806-a6f8-919f9a0a78b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:48.431797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33ee8d-d70b-4806-a6f8-919f9a0a78b5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:48.441412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 384/770] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 10 13:25:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:48.441412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6186b-64cb-4681-b430-f6fa99b26154 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.441412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6186b-64cb-4681-b430-f6fa99b26154 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.441412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6186b-64cb-4681-b430-f6fa99b26154 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.441412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 385/771] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:48.441412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad2a9d-6887-4e5b-a1a6-bb96bf5b2dfe None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.441860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad2a9d-6887-4e5b-a1a6-bb96bf5b2dfe None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.441860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad2a9d-6887-4e5b-a1a6-bb96bf5b2dfe None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.444054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 386/772] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:48.444054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad2a9d-6887-4e5b-a1a6-bb96bf5b2dfe None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:48.446497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ea277-8d71-4490-973a-bd0a5c97c786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:48.446632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ea277-8d71-4490-973a-bd0a5c97c786 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.446881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ea277-8d71-4490-973a-bd0a5c97c786 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.447154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ea277-8d71-4490-973a-bd0a5c97c786 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/4e395e059b5d436a8678bb302bcfafb5/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.449054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ea277-8d71-4490-973a-bd0a5c97c786 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=4e395e059b5d436a8678bb302bcfafb5, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:48.450551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ea277-8d71-4490-973a-bd0a5c97c786 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:48.465204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 387/773] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:25:48 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/4e395e059b5d436a8678bb302bcfafb5/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:48.466464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3d787-683d-4f39-b38b-7efd495ccd5b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.466742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3d787-683d-4f39-b38b-7efd495ccd5b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.467007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3d787-683d-4f39-b38b-7efd495ccd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.490739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9ba3d787-683d-4f39-b38b-7efd495ccd5b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:48.491976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9ba3d787-683d-4f39-b38b-7efd495ccd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:48.572732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 388/774] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:48.574123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f29f46-02c1-4b36-b313-79909cee16e3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.574403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f29f46-02c1-4b36-b313-79909cee16e3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.574673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f29f46-02c1-4b36-b313-79909cee16e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.603505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-75f29f46-02c1-4b36-b313-79909cee16e3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:48.606255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-75f29f46-02c1-4b36-b313-79909cee16e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:48.610909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad2a9d-6887-4e5b-a1a6-bb96bf5b2dfe None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:48.659196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:48.659739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.660018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.660286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.661637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] RBAC: Adding query filter params (interface=public, service_id=794df264d50c47ab9259ce7839cb62a3, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:48.661954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:48.663329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:48.672784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 387/775] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 10 13:25:48 2022] GET /identity/v3/endpoints?service_id=794df264d50c47ab9259ce7839cb62a3&interface=public®ion_id=RegionOne => generated 194 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:48.673586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:48.674247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:48.677992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:48.699198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 389/776] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:48.700398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f29f46-02c1-4b36-b313-79909cee16e3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:48.808329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0852e2-69a1-4d0f-ba80-319103294c5a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:48.819197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f29f46-02c1-4b36-b313-79909cee16e3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:48.853588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:48.854228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.854503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.854768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.855522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:48.856542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724ee94-5259-4b53-a13b-b510bf8f789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:48.857148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:48.857278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724ee94-5259-4b53-a13b-b510bf8f789e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:48.857460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724ee94-5259-4b53-a13b-b510bf8f789e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:48.857963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724ee94-5259-4b53-a13b-b510bf8f789e None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:48.858743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724ee94-5259-4b53-a13b-b510bf8f789e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:48.860250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724ee94-5259-4b53-a13b-b510bf8f789e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:48.866071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 390/777] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:25:48 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:48.866856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724ee94-5259-4b53-a13b-b510bf8f789e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:48.873868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:48.874942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:48.875789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 388/778] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:48 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:48.882040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:48.970240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:49.034388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:49.080888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724ee94-5259-4b53-a13b-b510bf8f789e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.093409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee092d-4608-4ba6-923a-0caa819877b1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.117978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e98847-1df2-4e6a-8be9-ccc22a89874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.118609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e98847-1df2-4e6a-8be9-ccc22a89874c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.118886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e98847-1df2-4e6a-8be9-ccc22a89874c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.119154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e98847-1df2-4e6a-8be9-ccc22a89874c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.129896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e98847-1df2-4e6a-8be9-ccc22a89874c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:49.134559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.135173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.135444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b9e98847-1df2-4e6a-8be9-ccc22a89874c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:49.135758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:49.137907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:49.140020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:49.141579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.141579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] RBAC: Adding query filter params (name=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:49.141579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.141992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 391/779] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:49.142132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.142667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e98847-1df2-4e6a-8be9-ccc22a89874c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.143154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 389/780] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:25:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:49.150635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.267989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e98847-1df2-4e6a-8be9-ccc22a89874c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.281865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:49.284480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42754fd1-c003-403d-90dc-d6ad4095fb6b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.315183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:49.322446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf696a-c973-4a56-986c-690191147f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.323052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf696a-c973-4a56-986c-690191147f60 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.323322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf696a-c973-4a56-986c-690191147f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.323587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf696a-c973-4a56-986c-690191147f60 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.323798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.324207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf696a-c973-4a56-986c-690191147f60 None admin] RBAC: Adding query filter params (type=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:49.324498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf696a-c973-4a56-986c-690191147f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.325033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.325544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.325885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf696a-c973-4a56-986c-690191147f60 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.326235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.327253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:49.327898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.329476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.335701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 390/781] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:49.336891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9d9d1-36f9-4fa7-a216-88c42a8c584a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.337171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9d9d1-36f9-4fa7-a216-88c42a8c584a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.337507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9d9d1-36f9-4fa7-a216-88c42a8c584a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.339750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 392/783] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:49.345069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.345191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 391/783] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:49.345191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684c4b3-0cdd-41a6-9572-5d8c8f3c59e3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.345191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684c4b3-0cdd-41a6-9572-5d8c8f3c59e3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.345191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684c4b3-0cdd-41a6-9572-5d8c8f3c59e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.363865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d684c4b3-0cdd-41a6-9572-5d8c8f3c59e3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:49.364951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d684c4b3-0cdd-41a6-9572-5d8c8f3c59e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:49.445370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 392/784] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:25:49.446262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684c4b3-0cdd-41a6-9572-5d8c8f3c59e3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.459589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e551-ef1c-452e-9cb1-faa35227a695 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.499742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.500355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.500631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.500899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.501843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] RBAC: Adding query filter params (interface=public, service_id=313ef7b1cbee45ee8648c963d7085887, region_id=RegionOne) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:49.502324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.504490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.513859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 393/785] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/endpoints?service_id=313ef7b1cbee45ee8648c963d7085887&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:49.514671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.554431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:49.591297 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684c4b3-0cdd-41a6-9572-5d8c8f3c59e3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.632444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:49.647528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.648186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.648464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.648735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.649369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378c9f4-85d0-4dcc-ac1c-492098285fd7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.660452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.662693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:49.678406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:49.679582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:49.680638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:49.680638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:49.680638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:49.680638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 393/786] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:49.680638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.696994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b298f2b-5367-440a-a6b3-adda67ef68c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.698341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b298f2b-5367-440a-a6b3-adda67ef68c6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.698769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b298f2b-5367-440a-a6b3-adda67ef68c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.699435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b298f2b-5367-440a-a6b3-adda67ef68c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.714984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b298f2b-5367-440a-a6b3-adda67ef68c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6b298f2b-5367-440a-a6b3-adda67ef68c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:49.721921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:49.723091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:49.724070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:49.731627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 394/787] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:49.731627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b298f2b-5367-440a-a6b3-adda67ef68c6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.814205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90b7ab-a766-45c2-9db2-ffc63912acc8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.831427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:49.842151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b298f2b-5367-440a-a6b3-adda67ef68c6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:49.862151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f259143-d84c-47d2-aa95-cc3cac3deb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.863552 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f259143-d84c-47d2-aa95-cc3cac3deb34 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.864873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f259143-d84c-47d2-aa95-cc3cac3deb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.865374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f259143-d84c-47d2-aa95-cc3cac3deb34 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.866481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f259143-d84c-47d2-aa95-cc3cac3deb34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.871985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f259143-d84c-47d2-aa95-cc3cac3deb34 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.885252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:49.885493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 394/788] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:49.885493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f259143-d84c-47d2-aa95-cc3cac3deb34 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.885642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:49.885642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:49.885642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:49.889257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:49.889257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:49.889257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:49.918658 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 395/789] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:49.923985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:49.958556 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.071334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f259143-d84c-47d2-aa95-cc3cac3deb34 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.102395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fc558-3260-4cb9-818c-93ccbfcd844d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.111571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.112348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.112631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.112976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.114696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:50.115007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.116475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.127624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 395/790] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:50.134162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:50.149117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.150385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.150869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.151739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.162976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.165156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:50.182500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:50.183570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:25:50.184493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:50.187971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 396/791] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:25:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:50.193863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:50.249079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec486a9e-72da-46f3-a96c-259f6cc4d0b3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.251572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.269621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.293002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.293711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.293952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.294221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.311847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.313323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:50.334387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:50.335358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:50.336238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:50.336238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:50.336238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:50.336238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 396/792] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:50.342128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:50.345831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4aa053-95df-4a82-9f76-71c260e0f0c1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.417949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.418564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.418843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.419110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.419840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:50.420138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.421786 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.453036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 397/793] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:50.462357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:50.466208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f6b42-972a-4d51-beb0-79b987ae20de None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.508870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.509609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.509886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.510154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.510801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:50.511099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.512468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.525944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 397/794] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:50.531108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:50.560469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.593685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.631743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca8041-cefd-4fa1-b223-6ebbe6cf2148 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.702529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bbc0e-03b8-476c-b71e-1abf0457a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.704006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bbc0e-03b8-476c-b71e-1abf0457a00f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.704488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bbc0e-03b8-476c-b71e-1abf0457a00f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.704988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bbc0e-03b8-476c-b71e-1abf0457a00f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.717978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bc61f-5cd0-4910-be1b-159398e279d0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.718727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bbc0e-03b8-476c-b71e-1abf0457a00f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.720693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bbc0e-03b8-476c-b71e-1abf0457a00f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-297bbc0e-03b8-476c-b71e-1abf0457a00f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:50.737571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:50.738539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:50.739479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:50.739479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:50.739479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:50.739479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 398/795] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:50.744854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6045-0c7d-4b2a-af8f-66488af5b3b1 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.745316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6045-0c7d-4b2a-af8f-66488af5b3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.745664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6045-0c7d-4b2a-af8f-66488af5b3b1 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.748782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 399/796] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:50.748782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6045-0c7d-4b2a-af8f-66488af5b3b1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:50.761303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3f2a4-eade-465d-a847-7115c237f1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.762191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3f2a4-eade-465d-a847-7115c237f1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.762620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3f2a4-eade-465d-a847-7115c237f1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.763038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3f2a4-eade-465d-a847-7115c237f1a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.773819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3f2a4-eade-465d-a847-7115c237f1a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.775536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3f2a4-eade-465d-a847-7115c237f1a9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-07c3f2a4-eade-465d-a847-7115c237f1a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:50.792982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:50.794021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:50.794957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:50.797656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 398/797] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:50.798648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b21f1-ffeb-4061-854e-b0c948ab866e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.798926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b21f1-ffeb-4061-854e-b0c948ab866e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.799190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b21f1-ffeb-4061-854e-b0c948ab866e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.816910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9f5b21f1-ffeb-4061-854e-b0c948ab866e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:50.818654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9f5b21f1-ffeb-4061-854e-b0c948ab866e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:50.878193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6045-0c7d-4b2a-af8f-66488af5b3b1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:50.905684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.906353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 399/798] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:50.907435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b21f1-ffeb-4061-854e-b0c948ab866e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:50.910900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:50.934544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:50.935149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:50.935422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:50.935694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:50.936326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:50.936617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:50.938289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:50.961270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 400/799] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:50.961270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.030109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b21f1-ffeb-4061-854e-b0c948ab866e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.066467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.067214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.068141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.068408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.069134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:51.069460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.070996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:51.089530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0570a1-0156-4459-a168-9280a5f90897 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.092706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 400/800] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:51.093571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.125982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a5940-acbf-4001-9772-b1af51dba2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.126608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a5940-acbf-4001-9772-b1af51dba2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.126926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a5940-acbf-4001-9772-b1af51dba2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.127200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a5940-acbf-4001-9772-b1af51dba2c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.138948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a5940-acbf-4001-9772-b1af51dba2c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-3b9a5940-acbf-4001-9772-b1af51dba2c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:51.143686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:51.144616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:51.145618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 401/801] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:51.146332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a5940-acbf-4001-9772-b1af51dba2c1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.173498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:51.200585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:51.229505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3182e9e-05a4-418e-ae13-66de3f692f78 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.268054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.268686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.268962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.269249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.277521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a5940-acbf-4001-9772-b1af51dba2c1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.279840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.281301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:51.292485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:51.293414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:51.294322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 401/802] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:51.294815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.318792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7cb1f-ee7b-4e7c-a648-7573fdc0b71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.319762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7cb1f-ee7b-4e7c-a648-7573fdc0b71f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.320278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7cb1f-ee7b-4e7c-a648-7573fdc0b71f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.320749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7cb1f-ee7b-4e7c-a648-7573fdc0b71f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.331389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7cb1f-ee7b-4e7c-a648-7573fdc0b71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.334409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7cb1f-ee7b-4e7c-a648-7573fdc0b71f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-27c7cb1f-ee7b-4e7c-a648-7573fdc0b71f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:51.349208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:51.350168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:51.351192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:51.351192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:51.351192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:51.356997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 402/803] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:51.358656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4166c-668a-4122-97e3-5327736c846a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.359088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4166c-668a-4122-97e3-5327736c846a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.359532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4166c-668a-4122-97e3-5327736c846a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.360916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 403/804] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:51.361795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4166c-668a-4122-97e3-5327736c846a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.403159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8058e4-a567-4b69-b2b9-d6a598ea20fd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.426593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:51.441665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.442676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.443565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.444050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.444922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:51.445765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.447898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:51.459151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 402/805] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:51.460728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.463517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:51.469915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4166c-668a-4122-97e3-5327736c846a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.504360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337622bb-be68-42e1-80f8-87bfb11e4463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.505016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337622bb-be68-42e1-80f8-87bfb11e4463 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.505437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337622bb-be68-42e1-80f8-87bfb11e4463 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.505798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337622bb-be68-42e1-80f8-87bfb11e4463 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.506771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337622bb-be68-42e1-80f8-87bfb11e4463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:51.507191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337622bb-be68-42e1-80f8-87bfb11e4463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.509255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337622bb-be68-42e1-80f8-87bfb11e4463 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:51.527739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 404/806] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:51.527739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc2dc-1677-43d4-bb77-77b2ca084244 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.527739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc2dc-1677-43d4-bb77-77b2ca084244 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.527739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc2dc-1677-43d4-bb77-77b2ca084244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.543557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d42fc2dc-1677-43d4-bb77-77b2ca084244 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:51.544941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d42fc2dc-1677-43d4-bb77-77b2ca084244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:51.577187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d6831-892d-4c82-b450-8a05c40d3bbd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.611804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.612409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.612677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.613153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.613599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:51.613896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.615268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:51.617792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 405/807] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:51.618435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fc2dc-1677-43d4-bb77-77b2ca084244 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.647874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 403/808] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:51.648572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.698702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:51.717185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:51.734940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fc2dc-1677-43d4-bb77-77b2ca084244 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.760167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a1d7d-9d6f-43ed-802f-bcf9efdef972 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.772099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.772728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.773009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.773303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.784369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.785886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:51.797525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb2db8-3253-4fac-9acb-60fb70d31a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.798417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb2db8-3253-4fac-9acb-60fb70d31a5a None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.798844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb2db8-3253-4fac-9acb-60fb70d31a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.799110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb2db8-3253-4fac-9acb-60fb70d31a5a None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/1e0811556a0542b685c98dabbaed5b81/roles/a9f22cef53e740cdbc09d1137362031e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:51.800854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:51.801902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:51.802920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:51.802920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:51.802920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:51.802920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 406/809] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:51.803601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.821092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb2db8-3253-4fac-9acb-60fb70d31a5a None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=1e0811556a0542b685c98dabbaed5b81, role_id=a9f22cef53e740cdbc09d1137362031e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.822576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb2db8-3253-4fac-9acb-60fb70d31a5a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:51.836566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 404/810] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:25:51 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/1e0811556a0542b685c98dabbaed5b81/roles/a9f22cef53e740cdbc09d1137362031e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:51.837120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb2db8-3253-4fac-9acb-60fb70d31a5a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:51.939758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf9f77-4bc1-42a0-9b72-120a38a89927 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.966230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:51.978677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb2db8-3253-4fac-9acb-60fb70d31a5a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:51.988890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:51.990449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:51.990928 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:51.992357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:51.992884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:51.994304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.000809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:52.010675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:52.011717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 13:25:52.012537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:52.015598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 407/811] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 10 13:25:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:52.015598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.035590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.036278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.036557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.036826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.043801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.045254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:52.057091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:52.058078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:52.058953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 405/812] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:52.059426 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.125628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160883c9-41b8-457e-b9f0-afeea7f5ccd0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.163120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.163727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.164004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.164123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef336ca1-8cc8-4bb6-ae2b-74ba171e80b8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.164283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.164908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:52.165248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.166928 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.183757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 408/813] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:52.183757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.203377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc50ab7-7d80-4024-90e2-94eb5aecdb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.204134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc50ab7-7d80-4024-90e2-94eb5aecdb7a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.204502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc50ab7-7d80-4024-90e2-94eb5aecdb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.204863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc50ab7-7d80-4024-90e2-94eb5aecdb7a None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.205770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc50ab7-7d80-4024-90e2-94eb5aecdb7a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:52.206163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc50ab7-7d80-4024-90e2-94eb5aecdb7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.207613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc50ab7-7d80-4024-90e2-94eb5aecdb7a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.213755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 406/814] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:52.215125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8c0f-6979-4d86-b9d9-450d50d67322 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.215561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8c0f-6979-4d86-b9d9-450d50d67322 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.215969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8c0f-6979-4d86-b9d9-450d50d67322 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.217725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 407/815] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:52.218679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea8c0f-6979-4d86-b9d9-450d50d67322 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.238160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:52.259411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:52.304208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff4b47-b1a9-4b67-a7ff-4fab3603e901 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.336254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea8c0f-6979-4d86-b9d9-450d50d67322 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.339149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.339569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.339858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.340131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.340819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:52.341120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.342584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.355384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 409/816] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:52.356193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.379033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33d9c0-8211-4a59-a812-181ea0e663f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.380472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d9c0-8211-4a59-a812-181ea0e663f0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.380810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d9c0-8211-4a59-a812-181ea0e663f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.381177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d9c0-8211-4a59-a812-181ea0e663f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.388469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33d9c0-8211-4a59-a812-181ea0e663f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.390022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33d9c0-8211-4a59-a812-181ea0e663f0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3d33d9c0-8211-4a59-a812-181ea0e663f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:52.403644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:52.404824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:52.406033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 408/817] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:52.406629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdbb88-a071-41dd-b4b8-4505eb4ac7a2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.406629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdbb88-a071-41dd-b4b8-4505eb4ac7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.406840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdbb88-a071-41dd-b4b8-4505eb4ac7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.426575 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-84cdbb88-a071-41dd-b4b8-4505eb4ac7a2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:52.428052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-84cdbb88-a071-41dd-b4b8-4505eb4ac7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:52.467482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86f823-29e3-456e-baa2-b7b06b5311a7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.490424 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:52.498087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 409/818] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:52.498848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdbb88-a071-41dd-b4b8-4505eb4ac7a2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.509599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.510271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.510610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.510979 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.511704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:52.512099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.513571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.519498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:52.520229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 410/819] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:52.520953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.619377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdbb88-a071-41dd-b4b8-4505eb4ac7a2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.633702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0d576-1124-4889-aaa5-f259c7b35e21 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.656125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55d74-0809-45fe-b98c-8891580a1db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.656805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55d74-0809-45fe-b98c-8891580a1db8 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.657083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55d74-0809-45fe-b98c-8891580a1db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.657403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55d74-0809-45fe-b98c-8891580a1db8 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/10209fea1bcc43509534b7e20f635c75/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.659299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e55d74-0809-45fe-b98c-8891580a1db8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=10209fea1bcc43509534b7e20f635c75, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.660724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e55d74-0809-45fe-b98c-8891580a1db8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.669758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.670466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.670734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.671164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.671826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:52.672117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.673536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.675227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 410/820] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:25:52 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/10209fea1bcc43509534b7e20f635c75/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:25:52.685642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55d74-0809-45fe-b98c-8891580a1db8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.710224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 411/821] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:52.711494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.768342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:52.777892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:52.828428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55d74-0809-45fe-b98c-8891580a1db8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.845796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d068d6-0c2e-4439-b75b-642d88c34232 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:52.874631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c92e9-1a9b-44ff-9633-021ecc117ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.875837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c92e9-1a9b-44ff-9633-021ecc117ac1 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.876285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c92e9-1a9b-44ff-9633-021ecc117ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.876689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c92e9-1a9b-44ff-9633-021ecc117ac1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.877853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c92e9-1a9b-44ff-9633-021ecc117ac1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.880936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c92e9-1a9b-44ff-9633-021ecc117ac1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.888398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:52.889031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:52.889337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:52.889613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] PATH_INFO: `/v3/services/network` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:52.890196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:52.891588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:52.902460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:52.904654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:52.906763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 412/822] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:52.908659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:52.919751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 411/823] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:25:52.921205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c92e9-1a9b-44ff-9633-021ecc117ac1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:53.058869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb3e3d-6409-4748-a7d4-089ef4410864 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.073611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:53.126013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c92e9-1a9b-44ff-9633-021ecc117ac1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.128345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa502d-80f9-47e8-bb34-d475dfb0bbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.129062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa502d-80f9-47e8-bb34-d475dfb0bbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.129482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa502d-80f9-47e8-bb34-d475dfb0bbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.130406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa502d-80f9-47e8-bb34-d475dfb0bbe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.149620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:53.169685 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.170308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.170592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.170863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.171500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] RBAC: Adding query filter params (name=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:53.171861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.173327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.183658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 412/824] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:53.184906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:53.206857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa502d-80f9-47e8-bb34-d475dfb0bbe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.208592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa502d-80f9-47e8-bb34-d475dfb0bbe1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.306207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9da0a-b47c-4c04-aba3-b7d83b346aaa None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.336401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 413/825] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:25:53.343120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa502d-80f9-47e8-bb34-d475dfb0bbe1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:53.350627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.351537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.356920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.357467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.358588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] RBAC: Adding query filter params (type=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:53.359117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.360702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.362494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:53.372759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 413/826] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:25:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:53.378928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:53.454595 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:53.470575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa502d-80f9-47e8-bb34-d475dfb0bbe1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.510358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4400f9d-c2c5-4bd7-aca9-3a50e464e385 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.514196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.515158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.515658 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.516142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.539797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.541695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.546201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a412c-c612-4bab-82b4-1cb3ad25ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.546806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a412c-c612-4bab-82b4-1cb3ad25ef4d None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.547078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a412c-c612-4bab-82b4-1cb3ad25ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.547350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a412c-c612-4bab-82b4-1cb3ad25ef4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.547992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a412c-c612-4bab-82b4-1cb3ad25ef4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.549494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a412c-c612-4bab-82b4-1cb3ad25ef4d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:53.568653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:53.569733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:53.571277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:53.571277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:53.571277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:53.571277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 414/827] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:53.575676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:53.587923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 414/828] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:25:53 2022] POST /identity/v3/endpoints => generated 337 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:53.649608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:53.691545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca43a60-4c08-4421-8821-5c287caab7ac None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.736449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.737167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.737475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.737744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.738561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:53.738857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.740892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.755896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 415/829] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:53.761674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a412c-c612-4bab-82b4-1cb3ad25ef4d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:53.877855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a412c-c612-4bab-82b4-1cb3ad25ef4d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:53.880869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:53.915285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0abf8-4a1d-4c4a-8da9-311d62418ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:53.915917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0abf8-4a1d-4c4a-8da9-311d62418ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:53.916195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0abf8-4a1d-4c4a-8da9-311d62418ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:53.916461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0abf8-4a1d-4c4a-8da9-311d62418ab3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:53.923392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0abf8-4a1d-4c4a-8da9-311d62418ab3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:53.924851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0abf8-4a1d-4c4a-8da9-311d62418ab3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6ed0abf8-4a1d-4c4a-8da9-311d62418ab3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:53.936066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:53.936974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:53.937914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 415/830] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:53.943531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.008332 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6cf0-9ba8-4d9e-af70-87305c4954ae None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.008651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6cf0-9ba8-4d9e-af70-87305c4954ae None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.008880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6cf0-9ba8-4d9e-af70-87305c4954ae None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.013412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 416/831] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:54.017019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f06861-c41f-43c7-ae14-4935e3ddaa74 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.017330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f06861-c41f-43c7-ae14-4935e3ddaa74 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.017606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f06861-c41f-43c7-ae14-4935e3ddaa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.054308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d2f06861-c41f-43c7-ae14-4935e3ddaa74 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:54.055523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d2f06861-c41f-43c7-ae14-4935e3ddaa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:54.079582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b337edb-bbac-42c3-9a61-e152e7f77ab5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.099148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:54.125669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:54.126800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.127561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.128306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.129214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:54.130255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:54.131870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:54.132245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 417/832] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:54.144618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 416/833] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:54.150785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f06861-c41f-43c7-ae14-4935e3ddaa74 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.156514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.250335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:54.257613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f06861-c41f-43c7-ae14-4935e3ddaa74 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.266545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c23db-62aa-4345-91ae-b100c5e7ec38 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.294128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:54.295006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.295497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.295959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.296886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=4e395e059b5d436a8678bb302bcfafb5) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:54.297436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:54.299105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:54.304162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc809c4-8c9e-4cfb-b4be-aa05926d0e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:54.304791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc809c4-8c9e-4cfb-b4be-aa05926d0e45 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.305067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc809c4-8c9e-4cfb-b4be-aa05926d0e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.305407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc809c4-8c9e-4cfb-b4be-aa05926d0e45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.306722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 418/834] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:54 2022] GET /identity/v3/role_assignments?user.id=4e395e059b5d436a8678bb302bcfafb5&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:54.316621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc809c4-8c9e-4cfb-b4be-aa05926d0e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8dc809c4-8c9e-4cfb-b4be-aa05926d0e45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:54.322336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:54.323361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:54.324191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 417/835] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:25:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:54.330092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.447803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554414-c450-433e-8746-eb56bb781d16 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.484937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:54.485585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.485861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.486134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.486791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:54.487152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:54.490402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:54.499205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 419/836] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:25:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:54.504698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc809c4-8c9e-4cfb-b4be-aa05926d0e45 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.542787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:54.613491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc809c4-8c9e-4cfb-b4be-aa05926d0e45 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.649279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:54.650277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.650775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.651237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.662563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:54.664254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:54.678384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:54.679311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:54.680187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:54.680187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:54.680187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:54.682238 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 418/837] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:54.690565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.712054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:54.796357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526efbc-91fb-432b-a1c8-ddd7cc104f41 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.832639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6961f93-cc1c-45fb-aa76-a97808f7899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:54.833672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961f93-cc1c-45fb-aa76-a97808f7899f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.834097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961f93-cc1c-45fb-aa76-a97808f7899f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.834543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961f93-cc1c-45fb-aa76-a97808f7899f None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.835332 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961f93-cc1c-45fb-aa76-a97808f7899f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:54.835786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961f93-cc1c-45fb-aa76-a97808f7899f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:54.837297 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961f93-cc1c-45fb-aa76-a97808f7899f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:54.850661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 420/838] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:54.856316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:54.949473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:54.958756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839c04-2353-4173-a13b-ec123f05f0f0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.959287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839c04-2353-4173-a13b-ec123f05f0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.959287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839c04-2353-4173-a13b-ec123f05f0f0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.960879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 421/839] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:54.965888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7bd0a-6443-4772-bc84-2803a389d35f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:54.966099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7bd0a-6443-4772-bc84-2803a389d35f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:54.966370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7bd0a-6443-4772-bc84-2803a389d35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:54.972532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1af4d-a622-42e0-94b5-3e2a123bf7bc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:54.985608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-51f7bd0a-6443-4772-bc84-2803a389d35f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:54.986793 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-51f7bd0a-6443-4772-bc84-2803a389d35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:55.013055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.013702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.013975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.014262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.026152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.028180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:55.043232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:55.044219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:25:55.045145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:55.047324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 419/840] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:25:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:55.052764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.077710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 422/841] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:25:55.097197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7bd0a-6443-4772-bc84-2803a389d35f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.116194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:55.176245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca99446-e7eb-4a3f-b34a-770dccf40abb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.213359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.214088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.214435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.214774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.215574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:55.216160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.217586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.231360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7bd0a-6443-4772-bc84-2803a389d35f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.241438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 420/842] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:55.243608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.250149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:55.274767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f05ff0-5fb7-4d55-952d-548ed6dd2f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.275463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f05ff0-5fb7-4d55-952d-548ed6dd2f3d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.275742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f05ff0-5fb7-4d55-952d-548ed6dd2f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.276014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f05ff0-5fb7-4d55-952d-548ed6dd2f3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.290268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f05ff0-5fb7-4d55-952d-548ed6dd2f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-07f05ff0-5fb7-4d55-952d-548ed6dd2f3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:55.295040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:55.295971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:25:55.296823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:55.302049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 423/843] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:25:55.303082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f05ff0-5fb7-4d55-952d-548ed6dd2f3d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.382183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:55.384574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbe109-7a45-4935-bde7-279a1a4ec239 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.416494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f05ff0-5fb7-4d55-952d-548ed6dd2f3d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.421852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d474605-a3b8-4481-94bc-9aac21e931cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.422514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d474605-a3b8-4481-94bc-9aac21e931cf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.422789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d474605-a3b8-4481-94bc-9aac21e931cf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.423059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d474605-a3b8-4481-94bc-9aac21e931cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.455217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafe10f-4a87-4096-bb90-6570e678228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.456331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe10f-4a87-4096-bb90-6570e678228d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.456858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe10f-4a87-4096-bb90-6570e678228d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.457397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe10f-4a87-4096-bb90-6570e678228d None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.458319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafe10f-4a87-4096-bb90-6570e678228d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:55.458818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafe10f-4a87-4096-bb90-6570e678228d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.460621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafe10f-4a87-4096-bb90-6570e678228d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.469524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 424/844] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:55.471016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49297-ac9c-4862-af43-f4bc4a400cb2 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.471517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49297-ac9c-4862-af43-f4bc4a400cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.471987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49297-ac9c-4862-af43-f4bc4a400cb2 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.474164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 425/845] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:55.479138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7159b-7515-47d8-baf7-76794d93635b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.479419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7159b-7515-47d8-baf7-76794d93635b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.479809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7159b-7515-47d8-baf7-76794d93635b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.496914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d474605-a3b8-4481-94bc-9aac21e931cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.498331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-14f7159b-7515-47d8-baf7-76794d93635b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:55.498443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d474605-a3b8-4481-94bc-9aac21e931cf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.499501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-14f7159b-7515-47d8-baf7-76794d93635b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:55.501846 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:55.572651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 426/846] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:55.573545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7159b-7515-47d8-baf7-76794d93635b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.619561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 421/847] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 13:25:55.620526 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d474605-a3b8-4481-94bc-9aac21e931cf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.644485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:55.684789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7159b-7515-47d8-baf7-76794d93635b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.721568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.722125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.722384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.722649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.728809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d474605-a3b8-4481-94bc-9aac21e931cf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.731642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:55.732991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.734590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:55.749590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:55.750582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:55.751490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:55.751490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:55.751490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:55.751490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 427/848] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:55.751490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.774189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.774805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.775078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.775342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.775910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.777292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:55.785796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:55.786804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:55.787636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 422/849] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:55.788468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.865181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ed074-941f-468d-b906-ad33ad9fd22b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.881842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:55.896878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f39fe-91ca-43e6-823d-39e397e8c507 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:55.904388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.905020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.905327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.905597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.916862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.918333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:55.933784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:55.934969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:55.935875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:55.935875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:55.935875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:55.935875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 428/850] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:55.936099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.936645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:55.937529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:55.937807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:55.938076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:55.938718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:55.939016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:55.940380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:55.954850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 423/851] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:55.955732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:55.986622 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.052687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca79a3c-ee39-4deb-b393-e242c1330687 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.065991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242592dc-cd16-4938-abb2-59f781b15e28 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.089531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.090132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.090405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.090666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.091279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:56.091572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.092930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.097853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 429/852] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:56.098490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.106825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.108415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.108895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.109732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.110578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:56.111391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.113396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.131398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 424/853] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:56.131398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.133638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.208647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3bbb-1a81-4a11-8317-86ad8c5b2b9c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.226985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.241814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79786c-6e42-45f2-a7d6-c2eae0f567d1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.242321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.242954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.243344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.243621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.254110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.256104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:25:56.268431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:56.269521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 430/854] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:56.270592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.285410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.286539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.286817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.287092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.287719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:56.288013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.289406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.293958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 425/855] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:56.294777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.404571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939d60-8247-473c-9d16-900e515dee1f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.429653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.430743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61682a-8a7c-4e9d-9c93-895c4cc5bf06 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.452404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.453553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.454395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.454677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.464157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.465966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.475873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cb223-a03c-4684-b6f6-e33e4e3cf0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.476474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb223-a03c-4684-b6f6-e33e4e3cf0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.476750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb223-a03c-4684-b6f6-e33e4e3cf0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.477034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb223-a03c-4684-b6f6-e33e4e3cf0e3 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.477800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cb223-a03c-4684-b6f6-e33e4e3cf0e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:56.478090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cb223-a03c-4684-b6f6-e33e4e3cf0e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:56.479250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:56.480111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:56.480997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:56.480997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:56.480997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:56.480997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:56.480997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:56.480997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:56.480997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:56.480997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:56.481877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cb223-a03c-4684-b6f6-e33e4e3cf0e3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.481949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 431/856] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:56.487843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.498829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 426/857] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:56.500004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652589-465d-4529-981e-f39d6d1bb547 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.500282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652589-465d-4529-981e-f39d6d1bb547 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.500553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652589-465d-4529-981e-f39d6d1bb547 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.502019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 427/858] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:56.505833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3652589-465d-4529-981e-f39d6d1bb547 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.519449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.601130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8f5a8-e9a8-4968-9927-87f6fd3caa32 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.614252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3652589-465d-4529-981e-f39d6d1bb547 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.636344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe66351-96bd-4246-b40e-9200835a0c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.636941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe66351-96bd-4246-b40e-9200835a0c5d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.637222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe66351-96bd-4246-b40e-9200835a0c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.637515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe66351-96bd-4246-b40e-9200835a0c5d None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.638318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe66351-96bd-4246-b40e-9200835a0c5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:56.638645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe66351-96bd-4246-b40e-9200835a0c5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.640134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe66351-96bd-4246-b40e-9200835a0c5d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.649720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c98dce-992b-4853-ab8f-9be26cd6b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.650325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98dce-992b-4853-ab8f-9be26cd6b334 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.650610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98dce-992b-4853-ab8f-9be26cd6b334 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.650884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98dce-992b-4853-ab8f-9be26cd6b334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.654345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 432/859] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:56.655703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5714f3f-b9ec-4b8f-bfda-a09fcda82015 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.655981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5714f3f-b9ec-4b8f-bfda-a09fcda82015 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.656246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5714f3f-b9ec-4b8f-bfda-a09fcda82015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.675020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a5714f3f-b9ec-4b8f-bfda-a09fcda82015 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:56.676202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a5714f3f-b9ec-4b8f-bfda-a09fcda82015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:56.679241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.727308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c98dce-992b-4853-ab8f-9be26cd6b334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.728959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c98dce-992b-4853-ab8f-9be26cd6b334 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.740914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.748553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 433/860] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:56.749294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5714f3f-b9ec-4b8f-bfda-a09fcda82015 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.852307 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 428/861] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 13:25:56.852307 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c98dce-992b-4853-ab8f-9be26cd6b334 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.869111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5714f3f-b9ec-4b8f-bfda-a09fcda82015 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.906846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:56.907455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:56.907730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:56.908002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:56.908709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=1e0811556a0542b685c98dabbaed5b81) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:56.909014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:56.910457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:56.916189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 434/862] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/role_assignments?user.id=1e0811556a0542b685c98dabbaed5b81&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:56.922181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:56.934670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:56.960784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c98dce-992b-4853-ab8f-9be26cd6b334 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:56.979086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:57.018688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.019356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.019636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.019908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.021166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.022635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:57.032582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:57.033660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:25:57.034561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:57.039082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 429/863] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:57.040202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.078032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3433d5-0731-4028-a612-a00f4d31eb92 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.117266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.118280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.118761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.119263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.132189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.133978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:57.149658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:57.150615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:57.151490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:57.151490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:57.151490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:57.153281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 435/864] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:57.155804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50a36-153f-4fe1-8232-c878de8d6d73 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.159658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.205445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.206054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.206329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.206595 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.207291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:57.207586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.209527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.216710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 430/865] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:57.222089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.233154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:57.280949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:57.295527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90c3d-540d-45fe-929c-da7000b4afec None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.340233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.341116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.341425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.341949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.342998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:57.343304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.345186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.349037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9238c-c0c6-4da4-bd1a-0e9007ad00d5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.366485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 436/866] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:57.371923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.392968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.393857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.394136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.394407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.395706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:57.396032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.397756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.405312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 431/867] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:25:57.505559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1eff4-d5bb-4bb4-bba2-5fef033a5779 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.525039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:57.545492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.546060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.546336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.546606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.555144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.556584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:25:57.568898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:57.570151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:57.571508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:25:57.571508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:25:57.571508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:25:57.571508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:25:57.571508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:25:57.571508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:25:57.571508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:25:57.571508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:57.575753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.576173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 437/868] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:57.688777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836b757-a5d3-48b3-affc-6ac142182f2c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.690685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:57.727269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81956b9e-37fd-4df9-bb54-cb215e7aec85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.728250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81956b9e-37fd-4df9-bb54-cb215e7aec85 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.728736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81956b9e-37fd-4df9-bb54-cb215e7aec85 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.729209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81956b9e-37fd-4df9-bb54-cb215e7aec85 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.730133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81956b9e-37fd-4df9-bb54-cb215e7aec85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:57.730660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81956b9e-37fd-4df9-bb54-cb215e7aec85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.732286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81956b9e-37fd-4df9-bb54-cb215e7aec85 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.745803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 432/869] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:57.756126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:57.862691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85f8b6-c5dc-41bd-aa8e-0daec2c9fcce None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:57.899137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c6b03-fbfa-49bd-bd69-f9e6fcff0e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:57.900184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c6b03-fbfa-49bd-bd69-f9e6fcff0e12 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:57.900674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c6b03-fbfa-49bd-bd69-f9e6fcff0e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:57.901144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c6b03-fbfa-49bd-bd69-f9e6fcff0e12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:57.902136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c6b03-fbfa-49bd-bd69-f9e6fcff0e12 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=10209fea1bcc43509534b7e20f635c75) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:57.902662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c6b03-fbfa-49bd-bd69-f9e6fcff0e12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:57.904306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c6b03-fbfa-49bd-bd69-f9e6fcff0e12 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:57.910788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 438/870] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:25:57 2022] GET /identity/v3/role_assignments?user.id=10209fea1bcc43509534b7e20f635c75&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:58.374970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c4495-d8dc-47ad-a6d7-881d54be90df None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.375516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c4495-d8dc-47ad-a6d7-881d54be90df None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.375516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c4495-d8dc-47ad-a6d7-881d54be90df None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.377059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 433/871] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:58.382942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7884b8-2720-4e39-b5fb-7d2676938aa3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.383386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7884b8-2720-4e39-b5fb-7d2676938aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.383487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7884b8-2720-4e39-b5fb-7d2676938aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.402247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ac7884b8-2720-4e39-b5fb-7d2676938aa3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:58.403438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ac7884b8-2720-4e39-b5fb-7d2676938aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:58.438668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:58.471792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 439/872] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:25:58.494080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c4495-d8dc-47ad-a6d7-881d54be90df None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:58.605260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c4495-d8dc-47ad-a6d7-881d54be90df None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:58.643463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c57d4-5d46-4771-9ce4-9d5755c35d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:58.644070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c57d4-5d46-4771-9ce4-9d5755c35d2c None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:58.644349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c57d4-5d46-4771-9ce4-9d5755c35d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:58.644617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c57d4-5d46-4771-9ce4-9d5755c35d2c None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:58.645365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c57d4-5d46-4771-9ce4-9d5755c35d2c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:58.646742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c57d4-5d46-4771-9ce4-9d5755c35d2c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:58.666655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 434/873] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:25:58 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:59.193568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8097438-b451-4a76-a7b4-995d1abcbd34 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.193977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8097438-b451-4a76-a7b4-995d1abcbd34 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.194095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8097438-b451-4a76-a7b4-995d1abcbd34 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.195733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 440/874] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:25:59.200830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3b36-9dc3-4fc7-8964-8e14ff891f2f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.201113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3b36-9dc3-4fc7-8964-8e14ff891f2f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.201419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3b36-9dc3-4fc7-8964-8e14ff891f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.219943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ecbb3b36-9dc3-4fc7-8964-8e14ff891f2f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:59.221107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ecbb3b36-9dc3-4fc7-8964-8e14ff891f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:59.239103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:59.302157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 435/875] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:25:59.325946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8097438-b451-4a76-a7b4-995d1abcbd34 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.431214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8097438-b451-4a76-a7b4-995d1abcbd34 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.466723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c8459-cbd1-4f40-86f9-88c65812f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.467283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c8459-cbd1-4f40-86f9-88c65812f57b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.467937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c8459-cbd1-4f40-86f9-88c65812f57b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.468208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c8459-cbd1-4f40-86f9-88c65812f57b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.478598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c8459-cbd1-4f40-86f9-88c65812f57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-654c8459-cbd1-4f40-86f9-88c65812f57b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:25:59.483702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:59.484609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:25:59.485473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 441/876] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 10 13:25:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:59.491528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb3b36-9dc3-4fc7-8964-8e14ff891f2f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.585867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ffed3-f917-4409-ad94-49f546fc4861 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.586189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ffed3-f917-4409-ad94-49f546fc4861 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.586414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ffed3-f917-4409-ad94-49f546fc4861 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.588034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 442/877] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:25:59.592935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8054-da5a-4a68-8526-63394c896b3f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.593215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8054-da5a-4a68-8526-63394c896b3f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.593518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8054-da5a-4a68-8526-63394c896b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.594388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb3b36-9dc3-4fc7-8964-8e14ff891f2f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.619753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fe7b8054-da5a-4a68-8526-63394c896b3f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:25:59.621256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fe7b8054-da5a-4a68-8526-63394c896b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:25:59.634352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.635001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.635285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.635549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.636215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:25:59.636511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.637926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.646807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 436/878] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 10 13:25:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:25:59.652347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.665104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:59.666539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:59.717472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 443/879] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:25:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:25:59.735044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8054-da5a-4a68-8526-63394c896b3f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.766027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1bbd4-2480-4c4e-80d6-f4d0a3ce0760 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.804172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.804889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.805166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.805461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.816901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.818878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:25:59.833759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:25:59.835365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:25:59.836550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:25:59.836550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:25:59.836550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:25:59.836550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 437/880] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:25:59.840541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:25:59.852294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8054-da5a-4a68-8526-63394c896b3f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:25:59.893128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e703f-b853-4435-a184-0d38e0bd16c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:25:59.893962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e703f-b853-4435-a184-0d38e0bd16c6 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:25:59.894328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e703f-b853-4435-a184-0d38e0bd16c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:25:59.894726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e703f-b853-4435-a184-0d38e0bd16c6 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:25:59.895460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e703f-b853-4435-a184-0d38e0bd16c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:25:59.897043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e703f-b853-4435-a184-0d38e0bd16c6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:25:59.905916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:25:59.923469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 444/881] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:25:59 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:25:59.960461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c445061-c6e9-4f75-9b8a-49deab9d44e4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.003549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d5f05-c0a1-4fef-b908-0bcf341e8dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.004080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d5f05-c0a1-4fef-b908-0bcf341e8dec None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.004352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d5f05-c0a1-4fef-b908-0bcf341e8dec None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.005144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d5f05-c0a1-4fef-b908-0bcf341e8dec None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.005817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d5f05-c0a1-4fef-b908-0bcf341e8dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:00.006111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d5f05-c0a1-4fef-b908-0bcf341e8dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.007486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d5f05-c0a1-4fef-b908-0bcf341e8dec None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.021118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ca8f-a8bb-4bb7-8e14-8c6cb76570b4 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.021453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ca8f-a8bb-4bb7-8e14-8c6cb76570b4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.021696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ca8f-a8bb-4bb7-8e14-8c6cb76570b4 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.023289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 445/882] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:00.025654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 438/883] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:00.027087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439ca8f-a8bb-4bb7-8e14-8c6cb76570b4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.028765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd446-5440-4a98-9a2a-d445a8760915 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.029046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd446-5440-4a98-9a2a-d445a8760915 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.029353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd446-5440-4a98-9a2a-d445a8760915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.047503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4cafd446-5440-4a98-9a2a-d445a8760915 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:00.048704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4cafd446-5440-4a98-9a2a-d445a8760915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.052669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.122069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 439/884] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:00.144059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafd446-5440-4a98-9a2a-d445a8760915 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.155305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439ca8f-a8bb-4bb7-8e14-8c6cb76570b4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.202400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.203408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.203891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.204412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.205158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.213448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.215184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:26:00.227260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:00.228238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:26:00.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:00.231772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 446/885] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:00.237925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.263065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafd446-5440-4a98-9a2a-d445a8760915 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.300436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.301057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.302054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.302278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.302862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.304327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:00.314103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:00.315193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:00.316436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 440/886] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:00.321508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.331597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.352695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54ec41-d594-4471-a870-40a1e5925e35 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.388999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.389891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.390348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.390766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.391665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:00.392143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.393801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.416483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 447/887] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:00.417982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.432983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498f771-58e3-4c95-93d9-5515e987457f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.447367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.472309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.473997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.474496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.475068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.476592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:00.476713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.478401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.485140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 441/888] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:00.490250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.526413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01363f90-cf4e-4af8-8e6b-d49f4dbfc318 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.563142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34a7bc-429d-4cf2-a479-17ce1c1e31ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.563995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34a7bc-429d-4cf2-a479-17ce1c1e31ea None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.564419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34a7bc-429d-4cf2-a479-17ce1c1e31ea None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.564856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34a7bc-429d-4cf2-a479-17ce1c1e31ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.574050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.601603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056333-dbaf-412c-9d74-f477042c4126 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.636748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34a7bc-429d-4cf2-a479-17ce1c1e31ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.638526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34a7bc-429d-4cf2-a479-17ce1c1e31ea None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.639158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b61d3e-4f2d-4c6f-8a64-1ad22caada4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.640011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b61d3e-4f2d-4c6f-8a64-1ad22caada4e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.640551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b61d3e-4f2d-4c6f-8a64-1ad22caada4e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.641106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b61d3e-4f2d-4c6f-8a64-1ad22caada4e None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.641957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b61d3e-4f2d-4c6f-8a64-1ad22caada4e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:00.642465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b61d3e-4f2d-4c6f-8a64-1ad22caada4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.644018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b61d3e-4f2d-4c6f-8a64-1ad22caada4e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.650169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 442/889] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:00.700966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927c78-3030-41a7-b39b-26f1374f647e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.701336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927c78-3030-41a7-b39b-26f1374f647e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.701534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927c78-3030-41a7-b39b-26f1374f647e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.703227 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 443/890] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:00.708997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05496379-9b50-480a-9e9b-8747bc2a09fa None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.709348 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05496379-9b50-480a-9e9b-8747bc2a09fa None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.709562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05496379-9b50-480a-9e9b-8747bc2a09fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.729341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-05496379-9b50-480a-9e9b-8747bc2a09fa None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:00.730665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-05496379-9b50-480a-9e9b-8747bc2a09fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:00.757755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 448/891] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 13:26:00.766064 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34a7bc-429d-4cf2-a479-17ce1c1e31ea None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.783101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.821621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:00.825048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 444/892] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:00.847267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05496379-9b50-480a-9e9b-8747bc2a09fa None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:00.904625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34a7bc-429d-4cf2-a479-17ce1c1e31ea None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.953907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:00.954538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:00.954863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:00.955135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:00.970322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05496379-9b50-480a-9e9b-8747bc2a09fa None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:00.971540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:00.974945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:00.993270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:00.994278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:00.995167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:00.995167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:00.995167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:01.001603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 449/893] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:01.003157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.013536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.014164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.014450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.014700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.015274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.017090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:01.029373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:01.030405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:26:01.031673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:01.035835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 445/894] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:26:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:01.037281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.081779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:01.113659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:01.125765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c09edf-2f62-4198-97b6-f0d988173372 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.169424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.170059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.170316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.170585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.171040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd568c5-16d8-449a-b2bd-426289842999 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.171219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:01.171518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.172907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.193261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 450/895] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:01.198605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.214356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.215087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.215476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.215818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.216670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:01.216906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.218405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.225287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 446/896] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:26:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:01.229034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.332126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:01.337827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ca239-6db8-4ff7-a757-403a90d94375 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.340164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159638d-46ee-4c3d-9af9-9d90d1266a09 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.374766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.375760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.376236 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.376753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.377646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:01.378233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.380309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.383573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.384485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.384769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.385039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.387137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 447/897] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:26:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:01.395885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.396028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.396108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:26:01.409114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:01.410075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:01.411747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 451/898] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:01.416932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.432622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:01.498088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9a1d0-aefa-42e2-ab21-72aacb597b54 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.535687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bc9e8-8728-4f9d-8873-3c1038de11e9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.542499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.543361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.544352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.544821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.546184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] RBAC: Adding query filter params (interface=public, service_id=c85ed3e78baf4284a9b4218b6e039fac, region_id=RegionOne) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:01.546706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.549019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.561094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 448/899] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 10 13:26:01 2022] GET /identity/v3/endpoints?service_id=c85ed3e78baf4284a9b4218b6e039fac&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:01.562612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.577370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.577990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.578266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.578587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.579587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:01.579935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.581722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.597353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 452/900] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:26:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:01.603791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:01.605901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:01.697447 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:01.707594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfebe1-012b-4346-8bb1-9764a834d2e3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.736233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a35de-954f-42ea-8a06-614e1f920879 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:01.751804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5a5bf-7662-40cf-9fd9-40afacc2f7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.753434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a5bf-7662-40cf-9fd9-40afacc2f7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.754073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a5bf-7662-40cf-9fd9-40afacc2f7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.755435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a5bf-7662-40cf-9fd9-40afacc2f7c7 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.756256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5a5bf-7662-40cf-9fd9-40afacc2f7c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.757852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5a5bf-7662-40cf-9fd9-40afacc2f7c7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.764287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 449/901] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:26:01 2022] GET /identity/v3/services => generated 1268 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:01.789023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662e83f-83ce-41b6-b931-368ba2f910ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:01.790052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e83f-83ce-41b6-b931-368ba2f910ec None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:01.790558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e83f-83ce-41b6-b931-368ba2f910ec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:01.791028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e83f-83ce-41b6-b931-368ba2f910ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:01.792065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662e83f-83ce-41b6-b931-368ba2f910ec None admin] RBAC: Adding query filter params (role.id=e583cc6231c94d0a849b9dbe1a2677b3, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=1e0811556a0542b685c98dabbaed5b81) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:01.792656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662e83f-83ce-41b6-b931-368ba2f910ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:01.794456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662e83f-83ce-41b6-b931-368ba2f910ec None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:01.802486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 453/902] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:01 2022] GET /identity/v3/role_assignments?user.id=1e0811556a0542b685c98dabbaed5b81&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=e583cc6231c94d0a849b9dbe1a2677b3 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:02.091540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e0a57-bf60-453c-8bb8-a7ee6caf3015 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.091861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e0a57-bf60-453c-8bb8-a7ee6caf3015 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.092103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e0a57-bf60-453c-8bb8-a7ee6caf3015 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.093693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 450/903] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:02.099290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e386b-d5f7-4b70-8927-03348c57f984 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.099561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e386b-d5f7-4b70-8927-03348c57f984 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.099831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e386b-d5f7-4b70-8927-03348c57f984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.118978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-883e386b-d5f7-4b70-8927-03348c57f984 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:02.120164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-883e386b-d5f7-4b70-8927-03348c57f984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:02.194175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 454/904] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:02.216042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e0a57-bf60-453c-8bb8-a7ee6caf3015 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:02.317131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:02.328146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e0a57-bf60-453c-8bb8-a7ee6caf3015 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:02.364641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:02.365176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.365476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.365744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.366327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:02.367685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:02.376626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:02.377643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:02.378587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 451/905] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 10 13:26:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:02.384131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e386b-d5f7-4b70-8927-03348c57f984 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:02.446946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:02.489184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e386b-d5f7-4b70-8927-03348c57f984 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:02.525185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:02.526089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.526530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.526951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.527723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:02.528170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:02.529687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:02.535059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 455/906] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:02.545266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:02.644797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226792-a58b-4dd5-a012-a9d357c91a13 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:02.679690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249d27d-b83b-42ab-be5c-bcf1ccbeebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:02.680347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249d27d-b83b-42ab-be5c-bcf1ccbeebf8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.680630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249d27d-b83b-42ab-be5c-bcf1ccbeebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.680898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249d27d-b83b-42ab-be5c-bcf1ccbeebf8 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.681557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249d27d-b83b-42ab-be5c-bcf1ccbeebf8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:02.681852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249d27d-b83b-42ab-be5c-bcf1ccbeebf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:02.683216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249d27d-b83b-42ab-be5c-bcf1ccbeebf8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:02.688433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 452/907] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:02.694333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:02.782637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14982d-9086-4ecf-8b69-0ed8689dccbb None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.783079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14982d-9086-4ecf-8b69-0ed8689dccbb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.783452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14982d-9086-4ecf-8b69-0ed8689dccbb None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.785124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 453/908] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:02.790767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0613-4dbc-476d-8641-6cc9715f6f60 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.791386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0613-4dbc-476d-8641-6cc9715f6f60 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.791970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0613-4dbc-476d-8641-6cc9715f6f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.804664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c397a0-50c4-4a5f-9170-8846fceca7d6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:02.811635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9a3b0613-4dbc-476d-8641-6cc9715f6f60 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:02.813046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9a3b0613-4dbc-476d-8641-6cc9715f6f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:02.829017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:02.835128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:02.845262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:02.846068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:02.846430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:02.846804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:02.847827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] RBAC: Adding query filter params (interface=public, service_id=01331a62ef8d401282706c3ff613ef61, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:02.848825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:02.850858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:02.857302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 456/909] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 10 13:26:02 2022] GET /identity/v3/endpoints?service_id=01331a62ef8d401282706c3ff613ef61&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:02.862765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:02.909386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 454/910] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:02.930117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b0613-4dbc-476d-8641-6cc9715f6f60 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:02.995297 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb528d-cb67-4f65-bec8-3280967bb65e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:03.033758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca6312-e95a-4c0f-bbf9-721ac7debf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:03.034841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6312-e95a-4c0f-bbf9-721ac7debf23 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.035336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6312-e95a-4c0f-bbf9-721ac7debf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.036446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6312-e95a-4c0f-bbf9-721ac7debf23 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.037299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca6312-e95a-4c0f-bbf9-721ac7debf23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:03.038985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca6312-e95a-4c0f-bbf9-721ac7debf23 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:03.049976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b0613-4dbc-476d-8641-6cc9715f6f60 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:03.050763 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 457/911] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:26:02 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:26:03.088497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e542f-17d4-42b7-b89c-844b225d8884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:03.089104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e542f-17d4-42b7-b89c-844b225d8884 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.089429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e542f-17d4-42b7-b89c-844b225d8884 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.089830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e542f-17d4-42b7-b89c-844b225d8884 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.090591 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e542f-17d4-42b7-b89c-844b225d8884 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:03.091987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e542f-17d4-42b7-b89c-844b225d8884 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:03.110280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 455/912] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:26:02 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:03.854968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c6661-197c-4041-b149-ee697e3afb90 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.855509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c6661-197c-4041-b149-ee697e3afb90 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.855936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c6661-197c-4041-b149-ee697e3afb90 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.857628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 458/913] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:03.863012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63f2f-a2c8-4b92-9277-2fd417a505c8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.863444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63f2f-a2c8-4b92-9277-2fd417a505c8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.863745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63f2f-a2c8-4b92-9277-2fd417a505c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.883019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c6a63f2f-a2c8-4b92-9277-2fd417a505c8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:03.884196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c6a63f2f-a2c8-4b92-9277-2fd417a505c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:03.886251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:03.937864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f7e5-9427-4f40-9525-e7c2271145d6 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.938200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f7e5-9427-4f40-9525-e7c2271145d6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.938404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f7e5-9427-4f40-9525-e7c2271145d6 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.940911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 459/914] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:03.945239 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c84ac-a7cc-4b3c-af8b-56ae7c621e4b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:03.945528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c84ac-a7cc-4b3c-af8b-56ae7c621e4b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:03.945803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c84ac-a7cc-4b3c-af8b-56ae7c621e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:03.958582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 456/915] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:03.964183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1e2c84ac-a7cc-4b3c-af8b-56ae7c621e4b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:03.965416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1e2c84ac-a7cc-4b3c-af8b-56ae7c621e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:03.981943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a63f2f-a2c8-4b92-9277-2fd417a505c8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.019330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:04.045401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 460/916] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:04.062046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c84ac-a7cc-4b3c-af8b-56ae7c621e4b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.087888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a63f2f-a2c8-4b92-9277-2fd417a505c8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.124820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.125485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.125820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.126018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.126593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.128056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.130752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:04.138210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:04.139235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 13:26:04.140477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:04.144452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 457/917] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:26:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:04.145363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.176238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c84ac-a7cc-4b3c-af8b-56ae7c621e4b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.216344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c1c9d-b3be-49f8-adf5-a73dcd21c7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.217452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c1c9d-b3be-49f8-adf5-a73dcd21c7ba None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.217957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c1c9d-b3be-49f8-adf5-a73dcd21c7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.218565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c1c9d-b3be-49f8-adf5-a73dcd21c7ba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.230629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c1c9d-b3be-49f8-adf5-a73dcd21c7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-770c1c9d-b3be-49f8-adf5-a73dcd21c7ba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:04.236283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:04.237250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:04.238117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:04.241276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 461/918] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:04.246859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c1c9d-b3be-49f8-adf5-a73dcd21c7ba None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.267564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3231b08-be78-4718-8f0e-807a6de0405d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.308640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:04.313952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.315432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.316088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.316655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.318712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:04.319593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.322001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.328765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 458/919] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:04.334304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.378870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c1c9d-b3be-49f8-adf5-a73dcd21c7ba None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.401506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:04.421411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.422089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.422412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.422679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.423382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:04.423681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.425028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.438253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 462/920] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:04.439099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.452723 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe857-91f8-4b18-aaa0-b090379ee2f0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.498547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.499724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.500356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.501107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.502322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:04.502827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.506195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.513669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 459/921] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:04.522105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.553145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429108ad-10a4-40af-b180-3d4a08ecc4e0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.565570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:04.595687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.596322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.596592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.596857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.608827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.610336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:04.625547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:04.626541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:04.627444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:04.627444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:04.627444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:04.632108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 463/922] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:04.632928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.651261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad45ed-360f-411d-af8b-1591e48e85b4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.677742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:04.687335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038d9d2-b2de-4975-b6e9-e9c64d190a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.688223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038d9d2-b2de-4975-b6e9-e9c64d190a0b None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.688223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038d9d2-b2de-4975-b6e9-e9c64d190a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.688521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038d9d2-b2de-4975-b6e9-e9c64d190a0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.689188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038d9d2-b2de-4975-b6e9-e9c64d190a0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.690854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038d9d2-b2de-4975-b6e9-e9c64d190a0b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.721324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 460/923] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:04.769830 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49895f5d-4d84-468b-9042-d663357457f7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:04.817825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:04.818904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:04.819438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:04.819913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:04.820826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:04.821416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:04.824491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:04.839514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 464/924] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:04.846534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038d9d2-b2de-4975-b6e9-e9c64d190a0b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:04.961828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038d9d2-b2de-4975-b6e9-e9c64d190a0b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.000125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d194aa-0ba0-42c6-b0a1-d664dcb5db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.000747 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d194aa-0ba0-42c6-b0a1-d664dcb5db7c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.001020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d194aa-0ba0-42c6-b0a1-d664dcb5db7c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.001320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d194aa-0ba0-42c6-b0a1-d664dcb5db7c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.009772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d194aa-0ba0-42c6-b0a1-d664dcb5db7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.011255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d194aa-0ba0-42c6-b0a1-d664dcb5db7c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-27d194aa-0ba0-42c6-b0a1-d664dcb5db7c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:26:05.022985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:05.023984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:05.024953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 461/925] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:26:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:05.030076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.049969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.131126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370169-c44f-467d-be5b-bbbff59b3c2d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.131669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370169-c44f-467d-be5b-bbbff59b3c2d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.131946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370169-c44f-467d-be5b-bbbff59b3c2d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.134118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 462/926] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:05.138975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809ffb3-2755-495c-ba19-a7e61ade0908 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.139300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809ffb3-2755-495c-ba19-a7e61ade0908 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.139427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648ddb-4927-476c-a3ac-7091f18ca5d5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.139518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809ffb3-2755-495c-ba19-a7e61ade0908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.158745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f809ffb3-2755-495c-ba19-a7e61ade0908 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:05.159956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f809ffb3-2755-495c-ba19-a7e61ade0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.175523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.176131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.176591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.176861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.177613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:05.177915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.179318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.198708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 465/927] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:05.204167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.205222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.244465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 463/928] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:05.246564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a0c80-e7bd-4df7-ba56-85d9d01faf7e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.247073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a0c80-e7bd-4df7-ba56-85d9d01faf7e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.247555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a0c80-e7bd-4df7-ba56-85d9d01faf7e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.249397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 464/929] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:05.259185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45447ba4-9fd2-40e4-987d-48e993889a9e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.259478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45447ba4-9fd2-40e4-987d-48e993889a9e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.259733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45447ba4-9fd2-40e4-987d-48e993889a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.281181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-45447ba4-9fd2-40e4-987d-48e993889a9e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:05.282414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-45447ba4-9fd2-40e4-987d-48e993889a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:05.295959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.323606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65510552-550a-4a68-b160-55038f2cfd21 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.353365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 465/930] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:05.354267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45447ba4-9fd2-40e4-987d-48e993889a9e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.363149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.364168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.364669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.365180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.376792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.378534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:05.393728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:05.394689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:05.395710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:05.395710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:05.395710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:05.399599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 466/931] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:05.400424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.473884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45447ba4-9fd2-40e4-987d-48e993889a9e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.475411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.503302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd2c2f-f441-443b-b0d8-f85cbd3d03f9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.511707 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.512432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.512703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.512967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.513676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.515132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:05.524655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:05.525901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:05.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 466/932] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:05.527619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.534010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.544751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.545523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.545795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.546067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.546661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.548007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:05.556732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:05.562836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:05.563950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 467/933] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:05.564926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.640994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf221-51e0-4f5e-8060-01386f38337f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.665321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc898-5849-4a87-a0c3-b345393f857c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.679211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.679809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.680089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.680360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.680997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:05.681327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.682700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.700184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 467/934] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:05.700184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.705956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.706917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.707189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.707452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.708495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:05.709054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.710593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.720780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 468/935] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:05.720780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.722487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.781431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.800374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715844a-4137-4a55-86e3-f4d70eff4b83 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.829433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bef7a-2f40-4f72-9a34-8cda2b1fdee6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.838620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.839225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.839514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.839756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.840573 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:05.840688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.842015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.851656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 468/936] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:05.851656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.870498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:05.871133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:05.871409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:05.871673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:05.878658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:05.880604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:26:05.891715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:05.892645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:05.893555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 469/937] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:05.894064 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:05.950526 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:05.961067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b989b-6a2d-429c-8bb1-5ac4f8a91be4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:05.998720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.000007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.000294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.000546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.001168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:06.001502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.002861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.012561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 469/938] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:06.012561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.017672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb34bb-dfa4-4703-914e-adf37073df24 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.033320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:06.060498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.061267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.061550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.061818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.062438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:06.062735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.064083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.069660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 470/939] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:06.070414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.113142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eac421-31cd-4163-b523-9015239e3541 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.159961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.161065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.161380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.161646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.162357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:06.162677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.164733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.176892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 470/940] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:06.177727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.192575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:06.193852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5e933-4d45-4a98-93ab-1f8aee26a7af None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.229378 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca00785-4148-4b7d-9744-b68e6f87e182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.230271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00785-4148-4b7d-9744-b68e6f87e182 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.230697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00785-4148-4b7d-9744-b68e6f87e182 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.231228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00785-4148-4b7d-9744-b68e6f87e182 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.232159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca00785-4148-4b7d-9744-b68e6f87e182 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.234109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca00785-4148-4b7d-9744-b68e6f87e182 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.260071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 471/941] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:06.261112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca00785-4148-4b7d-9744-b68e6f87e182 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.293966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b28f8-1f63-4e28-85f7-9e576ea2dc5b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.309536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:06.356525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.357501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.358060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.358338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.359063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] RBAC: Adding query filter params (interface=public, service_id=9dbff5b1bf014974bd5d71251423cffe, region_id=RegionOne) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:06.359371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.360843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.370554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 471/942] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/endpoints?service_id=9dbff5b1bf014974bd5d71251423cffe&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:06.376325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:06.410715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca00785-4148-4b7d-9744-b68e6f87e182 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.461527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddb7fd-a0f7-4b8e-a59b-f6baef18e47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.462510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddb7fd-a0f7-4b8e-a59b-f6baef18e47e None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.462977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddb7fd-a0f7-4b8e-a59b-f6baef18e47e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.463389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddb7fd-a0f7-4b8e-a59b-f6baef18e47e None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/1e0811556a0542b685c98dabbaed5b81/roles/e583cc6231c94d0a849b9dbe1a2677b3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.465774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddb7fd-a0f7-4b8e-a59b-f6baef18e47e None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=1e0811556a0542b685c98dabbaed5b81, role_id=e583cc6231c94d0a849b9dbe1a2677b3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.467202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddb7fd-a0f7-4b8e-a59b-f6baef18e47e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.468857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:06.483296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 472/943] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:26:06 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/1e0811556a0542b685c98dabbaed5b81/roles/e583cc6231c94d0a849b9dbe1a2677b3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:06.509123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe01b37-a382-4f2d-b03c-4a735e112a9d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:06.549314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef2028-349d-48ef-a35d-ebe4b8cf8847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:06.549857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef2028-349d-48ef-a35d-ebe4b8cf8847 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.550136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef2028-349d-48ef-a35d-ebe4b8cf8847 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.550407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef2028-349d-48ef-a35d-ebe4b8cf8847 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.551010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef2028-349d-48ef-a35d-ebe4b8cf8847 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:06.552942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef2028-349d-48ef-a35d-ebe4b8cf8847 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:06.559654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 472/944] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:26:06 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:06.871154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64cc03-53c1-44f4-b1a5-d7ab405929d7 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.871342 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64cc03-53c1-44f4-b1a5-d7ab405929d7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.871612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64cc03-53c1-44f4-b1a5-d7ab405929d7 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.873273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 473/945] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:06.880683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd395de3-436a-427c-8e9a-e789d6f58995 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:06.880955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd395de3-436a-427c-8e9a-e789d6f58995 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:06.881214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd395de3-436a-427c-8e9a-e789d6f58995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:06.900285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-dd395de3-436a-427c-8e9a-e789d6f58995 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:06.901489 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dd395de3-436a-427c-8e9a-e789d6f58995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:06.972056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 473/946] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:06.995958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64cc03-53c1-44f4-b1a5-d7ab405929d7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.096209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:07.108558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64cc03-53c1-44f4-b1a5-d7ab405929d7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.146272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.147260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.147749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.148221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.149085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.150814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.153718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING keystone.server.flask.common [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 13:26:07.173762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.sql.core [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 13:26:07.173762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 13:26:07.173762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [parameters: {'id': '0e60476044f348d0a4f6c17347329897', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 13:26:07.173762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80125) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e60476044f348d0a4f6c17347329897.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e60476044f348d0a4f6c17347329897. Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:26:07.182819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result.read() Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 13:26:07.183635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application yield resource Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise self.value Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application session.commit() Aug 10 13:26:07.184617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.session.flush() Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application compat.raise_( Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 13:26:07.185545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 13:26:07.187089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result.read() Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:26:07.187886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [parameters: {'id': '0e60476044f348d0a4f6c17347329897', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:07.188762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:07.189592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e60476044f348d0a4f6c17347329897. Aug 10 13:26:07.192883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:07.195939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 474/947] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Aug 10 13:26:06 2022] POST /identity/v3/projects => generated 262 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:07.202917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd395de3-436a-427c-8e9a-e789d6f58995 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.275576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:07.310808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd395de3-436a-427c-8e9a-e789d6f58995 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.344503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8d2b4-b96d-4d5e-be93-fb31fdacf30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.345130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8d2b4-b96d-4d5e-be93-fb31fdacf30f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.345424 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8d2b4-b96d-4d5e-be93-fb31fdacf30f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.345687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8d2b4-b96d-4d5e-be93-fb31fdacf30f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.352861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8d2b4-b96d-4d5e-be93-fb31fdacf30f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.354331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8d2b4-b96d-4d5e-be93-fb31fdacf30f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-0fe8d2b4-b96d-4d5e-be93-fb31fdacf30f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:26:07.365601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:07.366488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:07.367388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 474/948] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:07.372929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:07.480406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372036a-457c-48d6-bde4-38df85137c32 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:07.516155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067ccf-877c-41f8-88fc-f7afaa8cc600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:07.516769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067ccf-877c-41f8-88fc-f7afaa8cc600 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:07.517044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067ccf-877c-41f8-88fc-f7afaa8cc600 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:07.517339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067ccf-877c-41f8-88fc-f7afaa8cc600 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:07.518001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067ccf-877c-41f8-88fc-f7afaa8cc600 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:07.518300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067ccf-877c-41f8-88fc-f7afaa8cc600 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:07.519946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067ccf-877c-41f8-88fc-f7afaa8cc600 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:07.541074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 475/949] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:26:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:08.462380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9134987-290e-4f55-ba66-c852937a8065 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.462940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9134987-290e-4f55-ba66-c852937a8065 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.463181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9134987-290e-4f55-ba66-c852937a8065 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.464905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 475/950] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:08.470248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda3250-2731-4933-8ad0-742ff3eaccd6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.470657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda3250-2731-4933-8ad0-742ff3eaccd6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.470978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda3250-2731-4933-8ad0-742ff3eaccd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.490749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9bda3250-2731-4933-8ad0-742ff3eaccd6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:08.492245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9bda3250-2731-4933-8ad0-742ff3eaccd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:08.495118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:08.557467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b140a2-4c53-4de3-a1a4-54decea10c90 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.557753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b140a2-4c53-4de3-a1a4-54decea10c90 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.557936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b140a2-4c53-4de3-a1a4-54decea10c90 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.559504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 476/951] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:08.564309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a82e3a-1437-49f4-ba2c-a9675fabf345 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.564610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a82e3a-1437-49f4-ba2c-a9675fabf345 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.564880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a82e3a-1437-49f4-ba2c-a9675fabf345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.569358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 476/952] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:08.582990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-48a82e3a-1437-49f4-ba2c-a9675fabf345 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:08.584164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-48a82e3a-1437-49f4-ba2c-a9675fabf345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:08.585862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda3250-2731-4933-8ad0-742ff3eaccd6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.659990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 477/953] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:08.667862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618a861-a4f0-424a-9340-948490ddc7e2 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.668316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618a861-a4f0-424a-9340-948490ddc7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.668733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618a861-a4f0-424a-9340-948490ddc7e2 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.670674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 478/954] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:08.677093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a91d67-fcad-48aa-b16e-9526fdf3d43a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.677093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a91d67-fcad-48aa-b16e-9526fdf3d43a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.677093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a91d67-fcad-48aa-b16e-9526fdf3d43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.696519 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c4a91d67-fcad-48aa-b16e-9526fdf3d43a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:08.697713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c4a91d67-fcad-48aa-b16e-9526fdf3d43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:08.709212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda3250-2731-4933-8ad0-742ff3eaccd6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:08.746528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.747161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.747436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.747707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.748281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:08.749670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:08.756062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:08.758452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:08.761578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:08.762476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 477/955] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:08.763303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.765677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:08.777775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 479/956] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:08.778780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a91d67-fcad-48aa-b16e-9526fdf3d43a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.868991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194408-469c-4062-90d7-f6199de44a1b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:08.881439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a91d67-fcad-48aa-b16e-9526fdf3d43a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:08.905007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085af9c-e0f6-4813-898d-c79450b397b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.905669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085af9c-e0f6-4813-898d-c79450b397b6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.905946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085af9c-e0f6-4813-898d-c79450b397b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.906212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085af9c-e0f6-4813-898d-c79450b397b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.916237 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085af9c-e0f6-4813-898d-c79450b397b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:08.919119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:08.919723 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:08.919999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:08.920268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:08.920888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:08.921183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:08.922556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7085af9c-e0f6-4813-898d-c79450b397b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:08.922651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:08.923622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 478/957] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 10 13:26:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:08.924511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085af9c-e0f6-4813-898d-c79450b397b6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.929951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 480/958] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:26:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:08.930775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:08.986638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.006291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.044730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51610f-014e-43ed-9864-9fd8e4c2d844 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.056110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085af9c-e0f6-4813-898d-c79450b397b6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.080876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.081621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.081899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.082284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.083020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:09.083316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.085304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.090693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.091320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.091591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.091856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.092429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.093819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.098875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 481/959] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 10 13:26:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:09.098875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:09.107072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:09.108366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:09.109202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 479/960] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:09.110023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.205908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfcbcf-b65a-4d53-8ad5-0f5153c69198 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.210352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66546bfd-0163-4141-82d9-ee9d051315a6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.220774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.237965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.244291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.244896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.245168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.245461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.246101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:09.246437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.247562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.247738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.248189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.248462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.248732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.261257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 482/961] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:09.273255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.273492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.273492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:09.291379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:09.292603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:09.293733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:09.293733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:09.293733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:09.294301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 480/962] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:09.300389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.400677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5bb8-abb4-4617-9756-37e023bf2c1f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.445023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.445542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.445891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.446165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.446796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:09.447097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.448276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefde04e-b0b6-486d-89da-360c9127aa0a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.448443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.454733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 483/963] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:09.460981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.486854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.488112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.488539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.489473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.490315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:09.490761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.492305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.505459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 481/964] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:09.521259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.521433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.576842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.589184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0a42c-1f93-4b33-8448-a3645cbb056e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.634954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28280d14-5e06-44bf-80be-e134671e534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.635558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28280d14-5e06-44bf-80be-e134671e534b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.635831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28280d14-5e06-44bf-80be-e134671e534b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.636098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28280d14-5e06-44bf-80be-e134671e534b None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.636723 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28280d14-5e06-44bf-80be-e134671e534b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:09.637017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28280d14-5e06-44bf-80be-e134671e534b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.638419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28280d14-5e06-44bf-80be-e134671e534b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.645309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 484/965] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:09.645309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c4bc-bade-4529-9dd5-b26a0949b591 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.645309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c4bc-bade-4529-9dd5-b26a0949b591 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.645309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c4bc-bade-4529-9dd5-b26a0949b591 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.650028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 485/966] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:09.650028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9c4bc-bade-4529-9dd5-b26a0949b591 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.652434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d68ac-1c02-4237-a8d4-841c8aac4133 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.687611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2043e-2e5a-4d13-918e-4d60ff308e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.688262 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2043e-2e5a-4d13-918e-4d60ff308e6c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.688543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2043e-2e5a-4d13-918e-4d60ff308e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.688807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2043e-2e5a-4d13-918e-4d60ff308e6c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.696000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2043e-2e5a-4d13-918e-4d60ff308e6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.697466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2043e-2e5a-4d13-918e-4d60ff308e6c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f1b2043e-2e5a-4d13-918e-4d60ff308e6c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:26:09.709780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:09.710723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:09.711706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 482/967] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:09.712319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e4f66-ce05-4bf2-93b0-6fb1fe6af3e5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.712319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e4f66-ce05-4bf2-93b0-6fb1fe6af3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.712950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e4f66-ce05-4bf2-93b0-6fb1fe6af3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.730938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ce2e4f66-ce05-4bf2-93b0-6fb1fe6af3e5 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:09.732139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ce2e4f66-ce05-4bf2-93b0-6fb1fe6af3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:09.766894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9c4bc-bade-4529-9dd5-b26a0949b591 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.774735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.803717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e390c-c33c-4335-b296-e6eeabb46d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:09.804323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e390c-c33c-4335-b296-e6eeabb46d96 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:09.804595 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e390c-c33c-4335-b296-e6eeabb46d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:09.804858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e390c-c33c-4335-b296-e6eeabb46d96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:09.805364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 483/968] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:09.806203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e4f66-ce05-4bf2-93b0-6fb1fe6af3e5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.817471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e390c-c33c-4335-b296-e6eeabb46d96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:09.820028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e390c-c33c-4335-b296-e6eeabb46d96 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:09.836885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:09.853338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 486/969] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:09.853338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e390c-c33c-4335-b296-e6eeabb46d96 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:09.949594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e4f66-ce05-4bf2-93b0-6fb1fe6af3e5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:09.978228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e390c-c33c-4335-b296-e6eeabb46d96 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.002663 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.003725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.004094 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.004864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.006217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:10.006874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.009112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.017639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.017639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.017639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.018089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.018399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.020634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.022955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING keystone.server.flask.common [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 13:26:10.032876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 484/970] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 10 13:26:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:10.037596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.040706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.sql.core [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 13:26:10.040706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 13:26:10.040706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [parameters: {'user_id': '6d06137797024ce68f8dc3b434175893', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 13:26:10.040706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80124) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result.read() Aug 10 13:26:10.058643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:26:10.059657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application yield resource Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application next(self.gen) Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise self.value Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application session.commit() Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 13:26:10.060606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.session.flush() Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application compat.raise_( Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 13:26:10.061614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 13:26:10.062525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application conn.query(q) Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result.read() Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 13:26:10.063382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [parameters: {'user_id': '6d06137797024ce68f8dc3b434175893', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:10.064228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 13:26:10.065047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 13:26:10.069090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 13:26:10.069090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 13:26:10.069090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 13:26:10.069090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 13:26:10.069090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 13:26:10.069090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:10.069090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 487/971] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 10 13:26:09 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:10.069090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.092176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.175151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.204127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee6c60-d68a-49c1-a5a8-984c793e3806 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.219381 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0b68d-6fba-4323-a160-21c00ad4390c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.244916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aebf8-147a-4143-8b49-31b604798e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.248546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aebf8-147a-4143-8b49-31b604798e2c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.248546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aebf8-147a-4143-8b49-31b604798e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.248546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aebf8-147a-4143-8b49-31b604798e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.289872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.290892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.291405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.291869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.311099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.315535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:26:10.349292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:10.350300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 13:26:10.351256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:10.354008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 488/972] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:10.362659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.366879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568aebf8-147a-4143-8b49-31b604798e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.368561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568aebf8-147a-4143-8b49-31b604798e2c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.489654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.517205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.523049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d11b8-f8d0-4482-8741-ed9ca4f979ab None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.538572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 485/973] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 13:26:10.543968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aebf8-147a-4143-8b49-31b604798e2c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.619940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.620549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.620829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.621098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.621825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:10.622125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.624646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.701414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aebf8-147a-4143-8b49-31b604798e2c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.703116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 489/974] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:10.748033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.749084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.749612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.750085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.763097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.765099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:10.782185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:10.783177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:10.784197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:10.784197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:10.784197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:10.786607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 486/975] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:10.792688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.907614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc8c6b-d55a-436a-8684-8713fa763788 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.946273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.946890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.947169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.947446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.948098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:10.948395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.949801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.957455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.963333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 490/976] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:10.969270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:11.016684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:11.083084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba19476-fe92-45d7-bea1-fc989d615b51 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:11.120024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:11.120658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.120934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.121204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.127876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:11.129397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:26:11.140389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:11.141526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:11.142442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 487/977] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:11.148790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:11.257509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85feeffd-27a6-4a87-831d-db70401cb2aa None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:11.294581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87084ca2-6806-4151-a6f2-5916729569cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:11.295189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87084ca2-6806-4151-a6f2-5916729569cb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.295468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87084ca2-6806-4151-a6f2-5916729569cb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.295738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87084ca2-6806-4151-a6f2-5916729569cb None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.296426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87084ca2-6806-4151-a6f2-5916729569cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:11.296728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87084ca2-6806-4151-a6f2-5916729569cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:11.298187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87084ca2-6806-4151-a6f2-5916729569cb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:11.311242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 491/978] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:11.316986 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:11.384396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fc7ec-071a-467d-8547-faeb5ff7be60 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.384698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fc7ec-071a-467d-8547-faeb5ff7be60 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.384944 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fc7ec-071a-467d-8547-faeb5ff7be60 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.386437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 492/979] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:11.391079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288de5-1e1d-46fd-9453-5cf6ecc5a1c6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.391357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288de5-1e1d-46fd-9453-5cf6ecc5a1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.391623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288de5-1e1d-46fd-9453-5cf6ecc5a1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.411803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ac288de5-1e1d-46fd-9453-5cf6ecc5a1c6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:11.413335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ac288de5-1e1d-46fd-9453-5cf6ecc5a1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:11.417744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:11.421404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c9da9-1298-40ee-9a52-c450fed72f2f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:11.457792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:11.458741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.459013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.459393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.460124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] RBAC: Adding query filter params (role.id=e583cc6231c94d0a849b9dbe1a2677b3, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=1e0811556a0542b685c98dabbaed5b81) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:11.460432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:11.461877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:11.467861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 488/980] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:11 2022] GET /identity/v3/role_assignments?user.id=1e0811556a0542b685c98dabbaed5b81&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=e583cc6231c94d0a849b9dbe1a2677b3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:11.470811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:11.493884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 493/981] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:11.516534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:11.632807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd53642-d6fd-4f95-9130-1870b664c075 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:11.670493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99670c4b-d1e5-45ad-ac6f-698dd6171939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:11.671030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99670c4b-d1e5-45ad-ac6f-698dd6171939 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:11.671305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99670c4b-d1e5-45ad-ac6f-698dd6171939 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:11.671574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99670c4b-d1e5-45ad-ac6f-698dd6171939 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:11.672234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99670c4b-d1e5-45ad-ac6f-698dd6171939 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:11.673617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99670c4b-d1e5-45ad-ac6f-698dd6171939 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:11.688715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 489/982] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:26:11 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:12.000234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7259c-a395-4d5b-bab1-bbe24ffa1c2c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.000752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7259c-a395-4d5b-bab1-bbe24ffa1c2c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.000752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7259c-a395-4d5b-bab1-bbe24ffa1c2c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.002327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 494/983] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:12.007736 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a95b57-9b2f-4459-96fc-1cc61738da0c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.008022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a95b57-9b2f-4459-96fc-1cc61738da0c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.008293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a95b57-9b2f-4459-96fc-1cc61738da0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.027082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a2a95b57-9b2f-4459-96fc-1cc61738da0c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:12.028277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a2a95b57-9b2f-4459-96fc-1cc61738da0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:12.046518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.104863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 490/984] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:12.126975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7259c-a395-4d5b-bab1-bbe24ffa1c2c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:12.230428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7259c-a395-4d5b-bab1-bbe24ffa1c2c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:12.270525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:12.271067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.271339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.271605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.273300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:12.273577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:12.302463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:12.303559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:12.304475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:12.305315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 495/985] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:12.311691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a95b57-9b2f-4459-96fc-1cc61738da0c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:12.415233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a95b57-9b2f-4459-96fc-1cc61738da0c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:12.448584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:12.449183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.449478 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.449744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.450355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:12.450647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:12.451993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:12.457148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 491/986] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:12.463084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:12.516829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.564995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6c9d-5d59-4921-a472-3dad7d975920 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:12.601182 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:12.601872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.602147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.602413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.603053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:12.603341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:12.604777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:12.615898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 496/987] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:12.617181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:12.638244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.724129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06905c2e-335c-4625-91b0-678587e9b7f5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:12.757725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d9dff-d170-46e7-9e2d-852c1f57e35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:12.758328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9dff-d170-46e7-9e2d-852c1f57e35c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.758601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9dff-d170-46e7-9e2d-852c1f57e35c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.758869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9dff-d170-46e7-9e2d-852c1f57e35c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.759569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d9dff-d170-46e7-9e2d-852c1f57e35c None admin] RBAC: Adding query filter params (interface=admin, service_id=9dbff5b1bf014974bd5d71251423cffe, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:12.759871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d9dff-d170-46e7-9e2d-852c1f57e35c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:12.761250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d9dff-d170-46e7-9e2d-852c1f57e35c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:12.766425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 492/988] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Aug 10 13:26:12 2022] GET /identity/v3/endpoints?service_id=9dbff5b1bf014974bd5d71251423cffe&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:12.772462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:12.873343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a59a3-b57b-40cd-adb1-fa91ed676899 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:12.889589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:12.908854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf8f11-9fd7-41b2-9ef2-6a994047540b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:12.909482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf8f11-9fd7-41b2-9ef2-6a994047540b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:12.909759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf8f11-9fd7-41b2-9ef2-6a994047540b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:12.910026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf8f11-9fd7-41b2-9ef2-6a994047540b None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:12.910646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf8f11-9fd7-41b2-9ef2-6a994047540b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:12.912004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf8f11-9fd7-41b2-9ef2-6a994047540b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:12.918804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 497/989] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:26:12 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:13.524256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18450c-886e-4012-ac5a-56ea21db1bcf None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.524548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18450c-886e-4012-ac5a-56ea21db1bcf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.524951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18450c-886e-4012-ac5a-56ea21db1bcf None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.526432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 493/990] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:13.531415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733e3af-8150-4303-bf2a-515913f6c26b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.531868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733e3af-8150-4303-bf2a-515913f6c26b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.531955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733e3af-8150-4303-bf2a-515913f6c26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.550260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f733e3af-8150-4303-bf2a-515913f6c26b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:13.551508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f733e3af-8150-4303-bf2a-515913f6c26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:13.619150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 498/991] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:13.641102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18450c-886e-4012-ac5a-56ea21db1bcf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.723046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:13.743007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6426e-00d1-44a5-9acd-2ff3eb3fbd5e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.743323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6426e-00d1-44a5-9acd-2ff3eb3fbd5e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.743645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6426e-00d1-44a5-9acd-2ff3eb3fbd5e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.745030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 499/992] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:13.748176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18450c-886e-4012-ac5a-56ea21db1bcf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.749777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100a5a6-635f-40c6-8629-a8df4c02f16d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.750061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100a5a6-635f-40c6-8629-a8df4c02f16d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.750337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100a5a6-635f-40c6-8629-a8df4c02f16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.770676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d100a5a6-635f-40c6-8629-a8df4c02f16d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:13.771944 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d100a5a6-635f-40c6-8629-a8df4c02f16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:13.787854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.788790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.789341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.789788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] PATH_INFO: `/v3/services/image` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.790578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.792254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:13.802304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:13.803287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:13.804109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:13.805773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 494/993] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 10 13:26:13 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:13.811205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.847833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 500/994] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:13.870193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100a5a6-635f-40c6-8629-a8df4c02f16d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.912788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:13.917051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15281c85-7d8b-4c12-b9d4-6314b444cd09 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.954467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:13.955080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:13.955365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:13.955699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:13.956331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] RBAC: Adding query filter params (name=image) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:13.956645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:13.958533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:13.963471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 495/995] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:13.969030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:13.977577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100a5a6-635f-40c6-8629-a8df4c02f16d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:13.978957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:14.016538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.017223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.017633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.017897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.018558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.020005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:14.030809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:14.031900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:14.032743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 501/996] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:14.038742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:14.083289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134fce7-fdc4-4bc3-8828-b5d3f5f7d643 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.120612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.121222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.121520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.121793 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.122417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] RBAC: Adding query filter params (type=image) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:14.122715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.124062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.128471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 496/997] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:14.156268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19f529-6b9b-4758-9df7-a33a97ad73a7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.160090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:14.198377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.198995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.199272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.199541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.200186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:14.200484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.202001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.212825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 502/998] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:26:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:14.214246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:14.297404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:14.330285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b1636-cc24-45d0-ad81-a07ac405a408 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.365069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.365713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.365987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.366258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.366938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:14.367234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.368625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.373812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 497/999] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:26:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:14.379813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:14.479867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f000f2-a2dd-4bfd-86ce-44a14be3aa63 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.513909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e6a48-5246-4779-8947-176ee6b91ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.514516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e6a48-5246-4779-8947-176ee6b91ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.514790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e6a48-5246-4779-8947-176ee6b91ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.515055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e6a48-5246-4779-8947-176ee6b91ff4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.515747 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e6a48-5246-4779-8947-176ee6b91ff4 None admin] RBAC: Adding query filter params (interface=public, service_id=5e28949e4b674ee58b9f30de82d28419, region_id=RegionOne) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:14.516048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e6a48-5246-4779-8947-176ee6b91ff4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.517443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e6a48-5246-4779-8947-176ee6b91ff4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.526939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 503/1000] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 10 13:26:14 2022] GET /identity/v3/endpoints?service_id=5e28949e4b674ee58b9f30de82d28419&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:14.528588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:14.631351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c1959-b550-49c9-8726-946bd34a3d50 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:14.664536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965e5a-7ef2-4de7-b758-c6fb46daa61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:14.665157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965e5a-7ef2-4de7-b758-c6fb46daa61f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.665437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965e5a-7ef2-4de7-b758-c6fb46daa61f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.665698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965e5a-7ef2-4de7-b758-c6fb46daa61f None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.666295 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965e5a-7ef2-4de7-b758-c6fb46daa61f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:14.667714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965e5a-7ef2-4de7-b758-c6fb46daa61f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:14.668754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:14.681775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 498/1001] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:26:14 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:14.936585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c7b1-56e2-448c-802e-869ed7b65e6b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.937076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c7b1-56e2-448c-802e-869ed7b65e6b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.937076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c7b1-56e2-448c-802e-869ed7b65e6b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.938550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 504/1002] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:14.943489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5864a8-7c9d-4d1e-b12a-cb16bbae00db None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:14.943778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5864a8-7c9d-4d1e-b12a-cb16bbae00db None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:14.944052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5864a8-7c9d-4d1e-b12a-cb16bbae00db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:14.961732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fb5864a8-7c9d-4d1e-b12a-cb16bbae00db None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:14.962925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fb5864a8-7c9d-4d1e-b12a-cb16bbae00db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:15.031100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 499/1003] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:15.054157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298c7b1-56e2-448c-802e-869ed7b65e6b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:15.086243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:15.159311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298c7b1-56e2-448c-802e-869ed7b65e6b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:15.193328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:15.193897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:15.194175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:15.194445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:15.195029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:15.196395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:15.204988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:15.206147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 13:26:15.206988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:15.214638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 505/1004] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 10 13:26:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:15.214771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5864a8-7c9d-4d1e-b12a-cb16bbae00db None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:15.319200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5864a8-7c9d-4d1e-b12a-cb16bbae00db None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:15.345574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:15.355178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:15.355844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:15.356119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:15.356389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:15.357020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:15.357351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:15.358974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:15.368796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 500/1005] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:15.369919 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:15.463323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:15.470275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac76b3c-a3a2-433f-bf2a-ecf4c146bc9e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:15.503842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4b9a4-b3c5-4e4c-989f-8a4da54673df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:15.504485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b9a4-b3c5-4e4c-989f-8a4da54673df None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:15.504767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b9a4-b3c5-4e4c-989f-8a4da54673df None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:15.505035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b9a4-b3c5-4e4c-989f-8a4da54673df None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:15.505687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b9a4-b3c5-4e4c-989f-8a4da54673df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:15.505988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b9a4-b3c5-4e4c-989f-8a4da54673df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:15.507344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b9a4-b3c5-4e4c-989f-8a4da54673df None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:15.512177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 506/1006] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 10 13:26:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:15.517631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:15.616850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556270b-b273-45e6-8bc7-b2dd5f438bef None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:15.650165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaee9b-de3a-4b26-8182-8e7a5892d720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:15.650780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaee9b-de3a-4b26-8182-8e7a5892d720 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:15.651061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaee9b-de3a-4b26-8182-8e7a5892d720 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:15.651333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaee9b-de3a-4b26-8182-8e7a5892d720 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:15.651999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaee9b-de3a-4b26-8182-8e7a5892d720 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:15.653397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaee9b-de3a-4b26-8182-8e7a5892d720 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:15.677271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 501/1007] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:16.147375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46719f02-7312-4d3f-b51a-88334ce177f6 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.147926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46719f02-7312-4d3f-b51a-88334ce177f6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.147926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46719f02-7312-4d3f-b51a-88334ce177f6 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.149559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 507/1008] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:16.154452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd833e5-c344-40f6-839f-415279bafe1a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.154802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd833e5-c344-40f6-839f-415279bafe1a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.155263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd833e5-c344-40f6-839f-415279bafe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.173994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fcd833e5-c344-40f6-839f-415279bafe1a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:16.175160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fcd833e5-c344-40f6-839f-415279bafe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:16.224227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:16.256016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 502/1009] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:16.278203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46719f02-7312-4d3f-b51a-88334ce177f6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:16.380714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46719f02-7312-4d3f-b51a-88334ce177f6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:16.416342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5e388-c9e5-4444-ac86-c2a582fe0d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:16.416859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5e388-c9e5-4444-ac86-c2a582fe0d17 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.417130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5e388-c9e5-4444-ac86-c2a582fe0d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.417420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5e388-c9e5-4444-ac86-c2a582fe0d17 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.418057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5e388-c9e5-4444-ac86-c2a582fe0d17 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:16.419404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5e388-c9e5-4444-ac86-c2a582fe0d17 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:16.441297 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 508/1010] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:26:16 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:16.692919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f07107-66ac-4517-8f93-cd21448be682 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.693464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f07107-66ac-4517-8f93-cd21448be682 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.693464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f07107-66ac-4517-8f93-cd21448be682 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.694780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 503/1011] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:16.699979 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634fd7c-cfe7-4e0b-8eaa-fb69433d9f7d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.700269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634fd7c-cfe7-4e0b-8eaa-fb69433d9f7d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.700542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634fd7c-cfe7-4e0b-8eaa-fb69433d9f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.718720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e634fd7c-cfe7-4e0b-8eaa-fb69433d9f7d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:16.719903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e634fd7c-cfe7-4e0b-8eaa-fb69433d9f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:16.734948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:16.799281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 509/1012] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:16 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:16.823082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f07107-66ac-4517-8f93-cd21448be682 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:16.926635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f07107-66ac-4517-8f93-cd21448be682 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:16.960922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:16.961504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:16.961782 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:16.962047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:16.962620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:16.963974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:16.972566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:16.973778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 13:26:16.974628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:16.977548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 504/1013] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:16.979784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634fd7c-cfe7-4e0b-8eaa-fb69433d9f7d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.078567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634fd7c-cfe7-4e0b-8eaa-fb69433d9f7d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.112661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.113310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.114163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.114437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.115065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:17.115361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.116730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.121832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 510/1014] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:26:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:17.127613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.172006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:17.230633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc66f0e-6a3b-4ed5-95d1-e87146009925 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.268739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9cba9-9a55-436c-87d1-6f0a9579aee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.269386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cba9-9a55-436c-87d1-6f0a9579aee2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.269718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cba9-9a55-436c-87d1-6f0a9579aee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.270490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cba9-9a55-436c-87d1-6f0a9579aee2 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.275800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9cba9-9a55-436c-87d1-6f0a9579aee2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:17.275800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9cba9-9a55-436c-87d1-6f0a9579aee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.275800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9cba9-9a55-436c-87d1-6f0a9579aee2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.285467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 505/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 10 13:26:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:17.286683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.299214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:17.389003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09149ecc-45a6-4d0c-a681-e7c7b8db2079 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.423070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911e0c3-a7f2-4e03-964f-c362036235de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.423673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e0c3-a7f2-4e03-964f-c362036235de None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.423951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e0c3-a7f2-4e03-964f-c362036235de None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.424222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e0c3-a7f2-4e03-964f-c362036235de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.424866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911e0c3-a7f2-4e03-964f-c362036235de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.426291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911e0c3-a7f2-4e03-964f-c362036235de None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.453260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 511/1016] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:17.708227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bbc49-f8f5-4849-bf5f-5306220d753a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.708752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bbc49-f8f5-4849-bf5f-5306220d753a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.709017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bbc49-f8f5-4849-bf5f-5306220d753a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.710635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 506/1017] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:17.716084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b3146-f24a-4cab-978d-3ab7fa65df85 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.716374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b3146-f24a-4cab-978d-3ab7fa65df85 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.716647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b3146-f24a-4cab-978d-3ab7fa65df85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.734914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1a2b3146-f24a-4cab-978d-3ab7fa65df85 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:17.736412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1a2b3146-f24a-4cab-978d-3ab7fa65df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:17.797095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:17.812970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 512/1018] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:17.836052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bbc49-f8f5-4849-bf5f-5306220d753a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:17.940068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bbc49-f8f5-4849-bf5f-5306220d753a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:17.947686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:17.976618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcee6f-4958-4c05-acd5-954a2e35747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:17.977169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcee6f-4958-4c05-acd5-954a2e35747d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:17.977471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcee6f-4958-4c05-acd5-954a2e35747d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:17.977742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcee6f-4958-4c05-acd5-954a2e35747d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:17.978851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcee6f-4958-4c05-acd5-954a2e35747d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:17.980265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcee6f-4958-4c05-acd5-954a2e35747d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:17.986768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 507/1019] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:17.988007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b3146-f24a-4cab-978d-3ab7fa65df85 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:18.090160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b3146-f24a-4cab-978d-3ab7fa65df85 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:18.124156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8879039-27fb-4e7b-bde8-a2c0eec13fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:18.124757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8879039-27fb-4e7b-bde8-a2c0eec13fc6 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:18.125034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8879039-27fb-4e7b-bde8-a2c0eec13fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:18.125333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8879039-27fb-4e7b-bde8-a2c0eec13fc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:18.125984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8879039-27fb-4e7b-bde8-a2c0eec13fc6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:18.127472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8879039-27fb-4e7b-bde8-a2c0eec13fc6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:18.165260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 513/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:26:17 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:18.476603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5d565-f7e7-4a99-bc5a-92ff2d1246fc None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:18.476977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5d565-f7e7-4a99-bc5a-92ff2d1246fc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:18.477278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5d565-f7e7-4a99-bc5a-92ff2d1246fc None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:18.478769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 508/1021] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:18.483928 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029237c6-69a9-443f-b388-6a5b4a60dc28 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:18.484389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029237c6-69a9-443f-b388-6a5b4a60dc28 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:18.484466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029237c6-69a9-443f-b388-6a5b4a60dc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:18.502840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-029237c6-69a9-443f-b388-6a5b4a60dc28 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:18.504029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-029237c6-69a9-443f-b388-6a5b4a60dc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:18.543846 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:18.574599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 514/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:18.596428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5d565-f7e7-4a99-bc5a-92ff2d1246fc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:18.701887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5d565-f7e7-4a99-bc5a-92ff2d1246fc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:18.735691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:18.736339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:18.736579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:18.736848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:18.737442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:18.739344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:18.747987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:18.748999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:18.750198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:18.753318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 509/1023] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 10 13:26:18 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:18.755235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029237c6-69a9-443f-b388-6a5b4a60dc28 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:18.859264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029237c6-69a9-443f-b388-6a5b4a60dc28 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:18.892134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:18.892731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:18.893003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:18.893295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:18.893919 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] RBAC: Adding query filter params (name=image) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:18.894209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:18.895592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:18.905006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 515/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:18.906222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:18.947883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:19.008791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b8c6-78fa-4eb5-8697-388ee82f7de5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:19.042385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:19.042993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:19.043266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:19.043532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:19.044140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] RBAC: Adding query filter params (type=image) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:19.044438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:19.045814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:19.055489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 510/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:19.056997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:19.099601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:19.161471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e3cd-119e-43a8-887d-adda42c74a72 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:19.194912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680407e4-29fe-4fd6-aed4-c1d66922b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:19.195512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680407e4-29fe-4fd6-aed4-c1d66922b487 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:19.195790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680407e4-29fe-4fd6-aed4-c1d66922b487 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:19.196057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680407e4-29fe-4fd6-aed4-c1d66922b487 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:19.196748 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680407e4-29fe-4fd6-aed4-c1d66922b487 None admin] RBAC: Adding query filter params (interface=public, service_id=a337d0631b20492fb119d73e599167e1, region_id=RegionOne) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:19.197047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680407e4-29fe-4fd6-aed4-c1d66922b487 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:19.198451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680407e4-29fe-4fd6-aed4-c1d66922b487 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:19.208413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 516/1026] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 10 13:26:19 2022] GET /identity/v3/endpoints?service_id=a337d0631b20492fb119d73e599167e1&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:19.209577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:19.324222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39502173-df8d-4647-8a8d-3f2c3f7bfb1e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:19.325954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:19.360375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41fe5f-eefc-43da-9244-e45405175c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:19.360978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41fe5f-eefc-43da-9244-e45405175c7e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:19.361323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41fe5f-eefc-43da-9244-e45405175c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:19.361526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41fe5f-eefc-43da-9244-e45405175c7e None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:19.362123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41fe5f-eefc-43da-9244-e45405175c7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:19.364097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41fe5f-eefc-43da-9244-e45405175c7e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:19.377311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 511/1027] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:26:19 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:26:19.562178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc27c-1e53-49f8-8fb3-7084e6a38787 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:19.562670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc27c-1e53-49f8-8fb3-7084e6a38787 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:19.562670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bc27c-1e53-49f8-8fb3-7084e6a38787 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:19.564117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 517/1028] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:19.569665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea050d4a-29e5-44ca-b44c-df2605e68e1f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:19.569957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea050d4a-29e5-44ca-b44c-df2605e68e1f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:19.570234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea050d4a-29e5-44ca-b44c-df2605e68e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:19.589361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ea050d4a-29e5-44ca-b44c-df2605e68e1f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:19.590544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ea050d4a-29e5-44ca-b44c-df2605e68e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:19.658697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 512/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:19.681826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bc27c-1e53-49f8-8fb3-7084e6a38787 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:19.784286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bc27c-1e53-49f8-8fb3-7084e6a38787 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:19.785925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:19.820714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fd912-588d-4df6-af3a-4f8ac9153654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:19.821285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd912-588d-4df6-af3a-4f8ac9153654 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:19.823241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd912-588d-4df6-af3a-4f8ac9153654 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:19.823241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd912-588d-4df6-af3a-4f8ac9153654 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:19.832324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fd912-588d-4df6-af3a-4f8ac9153654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1a5fd912-588d-4df6-af3a-4f8ac9153654 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:19.836947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:19.838057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:19.838985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:19.841912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 518/1030] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 10 13:26:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:19.844320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea050d4a-29e5-44ca-b44c-df2605e68e1f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:19.938580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:19.945597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea050d4a-29e5-44ca-b44c-df2605e68e1f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:19.979845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:19.980455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:19.980732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:19.981003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:19.981693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:19.982015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:19.983634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:19.996349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 513/1031] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 10 13:26:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:19.998111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fd912-588d-4df6-af3a-4f8ac9153654 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:20.098008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fd912-588d-4df6-af3a-4f8ac9153654 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:20.131880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94329f-afc4-4c61-80ad-d9edd1119a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:20.132507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94329f-afc4-4c61-80ad-d9edd1119a2c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.132784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94329f-afc4-4c61-80ad-d9edd1119a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.133051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94329f-afc4-4c61-80ad-d9edd1119a2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.142867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94329f-afc4-4c61-80ad-d9edd1119a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:20.144306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94329f-afc4-4c61-80ad-d9edd1119a2c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6a94329f-afc4-4c61-80ad-d9edd1119a2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:20.158911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:20.165220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:20.166204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:20.166204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:20.166204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:20.166204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 519/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:20.166448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:20.212604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bded504-9319-4bc7-8167-add80f9ce059 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.213015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bded504-9319-4bc7-8167-add80f9ce059 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.213392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bded504-9319-4bc7-8167-add80f9ce059 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.214751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 520/1033] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:20.219697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfd7c1-4ef8-4775-bcf2-e381e3b73561 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.219981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfd7c1-4ef8-4775-bcf2-e381e3b73561 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.220253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfd7c1-4ef8-4775-bcf2-e381e3b73561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.238175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6edfd7c1-4ef8-4775-bcf2-e381e3b73561 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:20.239395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6edfd7c1-4ef8-4775-bcf2-e381e3b73561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:20.248670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:20.280791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac13f6-0ace-4d3c-a6f4-ea10c3a6432a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:20.313735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 521/1034] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:20.318571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:20.319420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.319926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.320398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.321273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:20.321777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:20.323344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:20.336417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 514/1035] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:20.339105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfd7c1-4ef8-4775-bcf2-e381e3b73561 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:20.341937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:20.360665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:20.443507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfd7c1-4ef8-4775-bcf2-e381e3b73561 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:20.460363 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eba4b-8ac7-412e-80ec-57ed823a26d7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:20.481665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31e285-71f6-4e69-a4cb-cc8905077bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:20.482407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31e285-71f6-4e69-a4cb-cc8905077bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.482921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31e285-71f6-4e69-a4cb-cc8905077bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.483192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31e285-71f6-4e69-a4cb-cc8905077bf1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.484286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31e285-71f6-4e69-a4cb-cc8905077bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:20.485954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31e285-71f6-4e69-a4cb-cc8905077bf1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:20.487930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 522/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:20.493634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31e285-71f6-4e69-a4cb-cc8905077bf1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:20.497864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:20.498497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.498774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.499044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.507157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:20.508588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:20.509047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:26:20.520467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:20.521536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:20.522843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 515/1037] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:26:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:20.527139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:20.604301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31e285-71f6-4e69-a4cb-cc8905077bf1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:20.605263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:20.627133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca9482-99c8-4a93-8b4e-e44d9fb0ef57 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:20.642026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ecf05-3c5a-45fb-9d44-64fb2021d3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:20.642761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ecf05-3c5a-45fb-9d44-64fb2021d3f3 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.643014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ecf05-3c5a-45fb-9d44-64fb2021d3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.643279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ecf05-3c5a-45fb-9d44-64fb2021d3f3 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.644072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ecf05-3c5a-45fb-9d44-64fb2021d3f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:20.646066 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ecf05-3c5a-45fb-9d44-64fb2021d3f3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:20.662891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:20.663595 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.663925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.664275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.665060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:20.665545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:20.667038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:20.685652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 523/1038] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:26:20 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:26:20.691268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 516/1039] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:26:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:20.693528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ecf05-3c5a-45fb-9d44-64fb2021d3f3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:20.765831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:20.804859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ecf05-3c5a-45fb-9d44-64fb2021d3f3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:20.840157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18d391-2d5a-4b52-8bc9-0645f43b975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:20.840760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d391-2d5a-4b52-8bc9-0645f43b975c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:20.841034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d391-2d5a-4b52-8bc9-0645f43b975c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:20.841327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d391-2d5a-4b52-8bc9-0645f43b975c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:20.911921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18d391-2d5a-4b52-8bc9-0645f43b975c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:20.913357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18d391-2d5a-4b52-8bc9-0645f43b975c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:20.978612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:21.014734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 524/1040] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:26:21.020813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:21.121461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e0112-b61a-4eb8-9cdf-3c4c85877ba4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:21.154255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:21.154884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.155249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.155518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.158131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:21.167327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:21.168679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:21.183227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:21.184190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:21.185175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:21.185175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:21.185175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:21.188970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 517/1041] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:21.190388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18d391-2d5a-4b52-8bc9-0645f43b975c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:21.298599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18d391-2d5a-4b52-8bc9-0645f43b975c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:21.333339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bb5e2-1dcf-4435-a932-54633fd6e4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:21.333941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb5e2-1dcf-4435-a932-54633fd6e4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.334212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb5e2-1dcf-4435-a932-54633fd6e4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.334476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb5e2-1dcf-4435-a932-54633fd6e4a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.335103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bb5e2-1dcf-4435-a932-54633fd6e4a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:21.335393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bb5e2-1dcf-4435-a932-54633fd6e4a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:21.336741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bb5e2-1dcf-4435-a932-54633fd6e4a3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:21.348147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 525/1042] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:21.353629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:21.371739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3185c53-2047-4672-b8a7-a68473fae46e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.372025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3185c53-2047-4672-b8a7-a68473fae46e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.372301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3185c53-2047-4672-b8a7-a68473fae46e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.373683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 526/1043] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:21.378730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edc508-cefa-4ab6-8c05-9964e0a4e166 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.379582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edc508-cefa-4ab6-8c05-9964e0a4e166 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.379880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edc508-cefa-4ab6-8c05-9964e0a4e166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.399013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-88edc508-cefa-4ab6-8c05-9964e0a4e166 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:21.400256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-88edc508-cefa-4ab6-8c05-9964e0a4e166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:21.424178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:21.465143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef87e47-9468-4bab-82ab-8bbc37da5c5e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:21.470697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 527/1044] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:21.493544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edc508-cefa-4ab6-8c05-9964e0a4e166 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:21.500183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:21.500878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.501202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.501539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.509644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:21.511087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:26:21.522084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:21.522982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:21.523873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 518/1045] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:21.528834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:21.580554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:21.597508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edc508-cefa-4ab6-8c05-9964e0a4e166 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:21.634858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:21.635486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.635588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc145e1-fad6-47e2-9814-c9511a06bf57 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:21.635798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.635986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] PATH_INFO: `/v3/services/image` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.636551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:21.638454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:21.646901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:21.647909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:21.648756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 528/1046] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 10 13:26:21 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:21.653894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:21.674081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:21.674739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.674958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.675220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.676445 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:21.676955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:21.678191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:21.679683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:21.695221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 519/1047] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:26:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:21.696265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:21.765551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0a3d-312d-4a57-a5f7-72a1edb13826 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:21.797726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c52499-f62e-4807-af90-5a4053bed7ee None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:21.803245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:21.804074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.804559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.805009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.805968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] RBAC: Adding query filter params (name=image) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:21.806459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:21.807463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:21.809181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:21.815423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 529/1048] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:21.821470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:21.838187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:21.838829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.839109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.839381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.840105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] RBAC: Adding query filter params (role.id=e583cc6231c94d0a849b9dbe1a2677b3, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=4e395e059b5d436a8678bb302bcfafb5) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:21.840413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:21.841887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:21.848258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 520/1049] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:21 2022] GET /identity/v3/role_assignments?user.id=4e395e059b5d436a8678bb302bcfafb5&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=e583cc6231c94d0a849b9dbe1a2677b3 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:21.927415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:21.945213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aacb87-81ff-45f8-b0e0-93bc4ab6bcae None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:21.981576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec6c51-1588-4e99-8956-b43f2f802f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:21.982499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6c51-1588-4e99-8956-b43f2f802f44 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:21.983202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6c51-1588-4e99-8956-b43f2f802f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:21.983680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6c51-1588-4e99-8956-b43f2f802f44 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:21.984547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec6c51-1588-4e99-8956-b43f2f802f44 None admin] RBAC: Adding query filter params (type=image) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:21.985025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec6c51-1588-4e99-8956-b43f2f802f44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:21.986692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec6c51-1588-4e99-8956-b43f2f802f44 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:21.993118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 530/1050] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:21.998817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:22.097929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e92e5-9f96-4df0-8db6-957d8808db03 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:22.131204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d433903-b8c6-4970-8047-0f41ac02ea81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:22.131815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d433903-b8c6-4970-8047-0f41ac02ea81 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:22.132090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d433903-b8c6-4970-8047-0f41ac02ea81 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:22.132354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d433903-b8c6-4970-8047-0f41ac02ea81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:22.133001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d433903-b8c6-4970-8047-0f41ac02ea81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:22.134392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d433903-b8c6-4970-8047-0f41ac02ea81 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:22.153812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 521/1051] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:26:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:22.766318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8776225-9375-478c-bd0d-f6478abb9a14 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:22.766849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8776225-9375-478c-bd0d-f6478abb9a14 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:22.766849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8776225-9375-478c-bd0d-f6478abb9a14 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:22.768497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 531/1052] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:22.774411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fa13-7417-41f9-a4b5-944bdaff60c0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:22.774702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fa13-7417-41f9-a4b5-944bdaff60c0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:22.775134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fa13-7417-41f9-a4b5-944bdaff60c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:22.794473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8587fa13-7417-41f9-a4b5-944bdaff60c0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:22.795681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8587fa13-7417-41f9-a4b5-944bdaff60c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:22.819697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:22.873940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 522/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:22.896826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8776225-9375-478c-bd0d-f6478abb9a14 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.001408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8776225-9375-478c-bd0d-f6478abb9a14 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.034454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7e934-6f54-4367-b913-d97752b7d471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.035004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7e934-6f54-4367-b913-d97752b7d471 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.035834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7e934-6f54-4367-b913-d97752b7d471 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.036103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7e934-6f54-4367-b913-d97752b7d471 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.046770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7e934-6f54-4367-b913-d97752b7d471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bad7e934-6f54-4367-b913-d97752b7d471 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:23.051347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:23.052457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:23.053518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:23.057330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 532/1054] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:23.058515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587fa13-7417-41f9-a4b5-944bdaff60c0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.171783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587fa13-7417-41f9-a4b5-944bdaff60c0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.205013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.205966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.206329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.206599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.207729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:23.208030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.209409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:23.218065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 523/1055] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:26:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:23.223558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7e934-6f54-4367-b913-d97752b7d471 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.236492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:23.331941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7e934-6f54-4367-b913-d97752b7d471 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.365434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bb631-7134-4076-9efb-b40ea4b029ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.366056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb631-7134-4076-9efb-b40ea4b029ff None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.366329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb631-7134-4076-9efb-b40ea4b029ff None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.366595 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb631-7134-4076-9efb-b40ea4b029ff None admin] PATH_INFO: `/v3/users/2de08724c36744c1a919ef8c8c0d8fc0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.380382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bb631-7134-4076-9efb-b40ea4b029ff None admin] RBAC: Authorizing `identity:get_user(user_id=2de08724c36744c1a919ef8c8c0d8fc0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.381881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bb631-7134-4076-9efb-b40ea4b029ff None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:23.388201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 533/1056] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:23 2022] GET /identity/v3/users/2de08724c36744c1a919ef8c8c0d8fc0 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:23.389211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.408766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:23.492890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e88a44-227f-4d21-8c8c-55b4c92d7fc3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.525978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e72cce-203a-42e9-a181-1ab11dd76015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.526577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e72cce-203a-42e9-a181-1ab11dd76015 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.526854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e72cce-203a-42e9-a181-1ab11dd76015 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.527122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e72cce-203a-42e9-a181-1ab11dd76015 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.593492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e72cce-203a-42e9-a181-1ab11dd76015 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.594856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e72cce-203a-42e9-a181-1ab11dd76015 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:23.611766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:23.698978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 524/1057] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:26:23.700245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bb631-7134-4076-9efb-b40ea4b029ff None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.767061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:23.802621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bb631-7134-4076-9efb-b40ea4b029ff None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.837321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c612f-3a96-47f9-abe0-ccda56dd828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.837947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c612f-3a96-47f9-abe0-ccda56dd828a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.838223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c612f-3a96-47f9-abe0-ccda56dd828a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.838492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c612f-3a96-47f9-abe0-ccda56dd828a None admin] PATH_INFO: `/v3/projects/309b82ea49ac4f668e791c60019886e8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.839608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c612f-3a96-47f9-abe0-ccda56dd828a None admin] RBAC: Authorizing `identity:get_project(project_id=309b82ea49ac4f668e791c60019886e8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.841033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c612f-3a96-47f9-abe0-ccda56dd828a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:23.848734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 534/1058] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:23 2022] GET /identity/v3/projects/309b82ea49ac4f668e791c60019886e8 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:23.849727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e72cce-203a-42e9-a181-1ab11dd76015 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:23.880567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3301fe-d724-48d8-b538-3dde2103e7fc None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.880870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3301fe-d724-48d8-b538-3dde2103e7fc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.881122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3301fe-d724-48d8-b538-3dde2103e7fc None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.882493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 535/1059] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:23.887757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4bdac-49f2-4e58-9796-4eb4c09695e7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.888033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4bdac-49f2-4e58-9796-4eb4c09695e7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.888302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4bdac-49f2-4e58-9796-4eb4c09695e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.906822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6fc4bdac-49f2-4e58-9796-4eb4c09695e7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:23.908229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6fc4bdac-49f2-4e58-9796-4eb4c09695e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:23.950419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e72cce-203a-42e9-a181-1ab11dd76015 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:23.967659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:23.985273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 536/1060] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:23.988914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:23.989551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:23.989824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:23.990087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:23.990800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=309b82ea49ac4f668e791c60019886e8, user.id=2de08724c36744c1a919ef8c8c0d8fc0) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:23.991100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:23.992506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.001275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 525/1061] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:23 2022] GET /identity/v3/role_assignments?user.id=2de08724c36744c1a919ef8c8c0d8fc0&scope.project.id=309b82ea49ac4f668e791c60019886e8&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:24.007204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4bdac-49f2-4e58-9796-4eb4c09695e7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.082297 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.121444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4bdac-49f2-4e58-9796-4eb4c09695e7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.157432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e267589-aff1-48d0-8cdf-7b47fd273cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.158063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e267589-aff1-48d0-8cdf-7b47fd273cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.158341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e267589-aff1-48d0-8cdf-7b47fd273cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.158612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e267589-aff1-48d0-8cdf-7b47fd273cd4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.169989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e267589-aff1-48d0-8cdf-7b47fd273cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1e267589-aff1-48d0-8cdf-7b47fd273cd4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:24.174609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:24.175559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:24.177026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:24.178398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 537/1062] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:24.182415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.194614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a93bf-d830-495d-ac47-2ec48abf0f9e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.194899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a93bf-d830-495d-ac47-2ec48abf0f9e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.195165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a93bf-d830-495d-ac47-2ec48abf0f9e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.196832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 538/1063] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:24.202682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06447-c635-4bd3-bfd3-b58b0fa325b5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.202960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06447-c635-4bd3-bfd3-b58b0fa325b5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.203227 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06447-c635-4bd3-bfd3-b58b0fa325b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.222360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-faa06447-c635-4bd3-bfd3-b58b0fa325b5 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:24.224162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-faa06447-c635-4bd3-bfd3-b58b0fa325b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:24.297261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 539/1064] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:24.304345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2d5ed-233a-4223-b373-3f4de65a04d7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.316528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa06447-c635-4bd3-bfd3-b58b0fa325b5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.341497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.342064 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.342366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.342638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.343233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:24.343523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.344917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.353858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 526/1065] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:24.360379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.379474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.405014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.430474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa06447-c635-4bd3-bfd3-b58b0fa325b5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.465972 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b80ca-7ecf-47d5-8624-935d18dbee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.466303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49696ff5-3edf-40d0-989e-66bfdbd6843b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.466575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b80ca-7ecf-47d5-8624-935d18dbee94 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.466849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b80ca-7ecf-47d5-8624-935d18dbee94 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.467114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b80ca-7ecf-47d5-8624-935d18dbee94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.502094 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.502723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.503028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.503272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.514298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.515771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:24.531492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:24.532546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:24.533472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:24.533472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:24.533472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:24.533472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 527/1066] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:24.538930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.540675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b80ca-7ecf-47d5-8624-935d18dbee94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.543003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b80ca-7ecf-47d5-8624-935d18dbee94 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.604850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.637722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.647257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee9fb-03e4-4432-96bb-69014cb6ce55 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.657207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 540/1067] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 13:26:24.658375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b80ca-7ecf-47d5-8624-935d18dbee94 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.687458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.688072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.688350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.688621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.689280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:24.689584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.690940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.703532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 528/1068] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:24.709911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.767345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b80ca-7ecf-47d5-8624-935d18dbee94 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.803430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.804432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.804709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.804976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.812672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b411b-5ab6-400b-b9ac-06a6a84edb39 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.815857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.817850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:24.833286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:24.834242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:24.835160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:24.835160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:24.835160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:24.835160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 541/1069] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:24.840220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.850317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.850944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.851218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.851479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.853605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.859977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.861507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:26:24.872695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:24.873681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:24.874604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 529/1070] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:24.879991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:24.897130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:24.953791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d7692-e7b9-493a-809b-a1c796b51b30 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.986742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1916fe4-4fa8-4855-a1af-7bcd134f20a4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:24.990352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:24.990948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:24.991223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:24.991491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:24.992126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:24.992422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:24.993818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.017260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 542/1071] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:25.034535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.035198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.035476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.035792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.036548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:25.036850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.038348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.058912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 530/1072] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:26:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:25.064987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.150131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:25.198175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3205e58-5038-4965-a334-55612ea8cf72 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.234930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.235555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.235827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.236091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.250347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.251861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:25.267631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:25.268596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:25.269594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:25.269594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:25.269594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:25.269594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 543/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:25.277511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.350504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:25.380396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fcfdc-9378-462f-a6cf-5fb1b1db23f1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.413076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.413640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.413918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.414188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.414820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:25.415120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.416477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.428932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 531/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:25.434548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.531860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7420a9-ac1e-4a9a-b80f-e6d57a3f41df None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.552046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:25.566940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.567557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.567834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.568161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.574728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.576160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:26:25.586888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:25.588006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:25.589146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 544/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:25.593744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.690588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d08d3-8c68-4958-948c-4a70a237ca23 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.718964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:25.724804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb40219-ff4e-43db-b84b-7553719a6032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.725438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb40219-ff4e-43db-b84b-7553719a6032 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.725718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb40219-ff4e-43db-b84b-7553719a6032 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.725987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb40219-ff4e-43db-b84b-7553719a6032 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.726692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb40219-ff4e-43db-b84b-7553719a6032 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:25.726976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb40219-ff4e-43db-b84b-7553719a6032 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.728387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb40219-ff4e-43db-b84b-7553719a6032 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.740201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 532/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:26:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:25.746081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:25.843194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc432351-7499-4070-b48c-1868c88aa0c7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:25.876048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68607e-ee9b-4474-9fa1-023ab32e6f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:25.876693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68607e-ee9b-4474-9fa1-023ab32e6f53 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.876969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68607e-ee9b-4474-9fa1-023ab32e6f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.877255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68607e-ee9b-4474-9fa1-023ab32e6f53 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/4e395e059b5d436a8678bb302bcfafb5/roles/e583cc6231c94d0a849b9dbe1a2677b3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.879348 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68607e-ee9b-4474-9fa1-023ab32e6f53 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=4e395e059b5d436a8678bb302bcfafb5, role_id=e583cc6231c94d0a849b9dbe1a2677b3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:25.880768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68607e-ee9b-4474-9fa1-023ab32e6f53 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:25.894225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 545/1077] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:26:25 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/4e395e059b5d436a8678bb302bcfafb5/roles/e583cc6231c94d0a849b9dbe1a2677b3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:26:25.989447 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e41991-f79e-4847-b594-d894858b3973 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.989631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e41991-f79e-4847-b594-d894858b3973 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.989962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e41991-f79e-4847-b594-d894858b3973 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:25.991351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 533/1078] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:25.996981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39b3cc-bea8-47c0-a8b4-dd5c27f1200c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:25.997306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39b3cc-bea8-47c0-a8b4-dd5c27f1200c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:25.997584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39b3cc-bea8-47c0-a8b4-dd5c27f1200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.015977 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-cf39b3cc-bea8-47c0-a8b4-dd5c27f1200c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:26.017198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cf39b3cc-bea8-47c0-a8b4-dd5c27f1200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:26.055828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:26.089274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 546/1079] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:26.111515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e41991-f79e-4847-b594-d894858b3973 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.211076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e41991-f79e-4847-b594-d894858b3973 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.248649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277adc7c-1f91-4b79-84c4-788a1667dcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.248649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adc7c-1f91-4b79-84c4-788a1667dcbb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.248649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adc7c-1f91-4b79-84c4-788a1667dcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.248649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adc7c-1f91-4b79-84c4-788a1667dcbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.258989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277adc7c-1f91-4b79-84c4-788a1667dcbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-277adc7c-1f91-4b79-84c4-788a1667dcbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:26.263670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:26.264583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:26.265453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 534/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:26.271905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39b3cc-bea8-47c0-a8b4-dd5c27f1200c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.370607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39b3cc-bea8-47c0-a8b4-dd5c27f1200c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.402889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.403487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.403763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.404032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.404687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:26.404985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.406380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.415018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 547/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:26.420541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277adc7c-1f91-4b79-84c4-788a1667dcbb None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.474149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:26.520675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277adc7c-1f91-4b79-84c4-788a1667dcbb None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.553679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d666bd6-a0a8-4191-95a9-beef53e9af03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.554303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d666bd6-a0a8-4191-95a9-beef53e9af03 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.554579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d666bd6-a0a8-4191-95a9-beef53e9af03 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.554849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d666bd6-a0a8-4191-95a9-beef53e9af03 None admin] PATH_INFO: `/v3/users/2de08724c36744c1a919ef8c8c0d8fc0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.555954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d666bd6-a0a8-4191-95a9-beef53e9af03 None admin] RBAC: Authorizing `identity:get_user(user_id=2de08724c36744c1a919ef8c8c0d8fc0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.557421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d666bd6-a0a8-4191-95a9-beef53e9af03 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.559210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 535/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/users/2de08724c36744c1a919ef8c8c0d8fc0 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:26.564682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.595272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:26.661859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad00a49-2a48-465f-b1a7-1b827e4e1b3b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.693135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ab62f-d5b1-4870-9b0a-0eef0dc8e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.693799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ab62f-d5b1-4870-9b0a-0eef0dc8e1fd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.694069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ab62f-d5b1-4870-9b0a-0eef0dc8e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.694330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ab62f-d5b1-4870-9b0a-0eef0dc8e1fd None admin] PATH_INFO: `/v3/projects/309b82ea49ac4f668e791c60019886e8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.695412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ab62f-d5b1-4870-9b0a-0eef0dc8e1fd None admin] RBAC: Authorizing `identity:get_project(project_id=309b82ea49ac4f668e791c60019886e8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.696813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ab62f-d5b1-4870-9b0a-0eef0dc8e1fd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.698440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 548/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:26 2022] GET /identity/v3/projects/309b82ea49ac4f668e791c60019886e8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:26.704479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d666bd6-a0a8-4191-95a9-beef53e9af03 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:26.804863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d666bd6-a0a8-4191-95a9-beef53e9af03 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:26.824256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:26.838873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70b955-ac8e-4be6-b8f2-10e09dafea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:26.839524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b955-ac8e-4be6-b8f2-10e09dafea55 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:26.839801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b955-ac8e-4be6-b8f2-10e09dafea55 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:26.840070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b955-ac8e-4be6-b8f2-10e09dafea55 None admin] PATH_INFO: `/v3/projects/309b82ea49ac4f668e791c60019886e8/users/2de08724c36744c1a919ef8c8c0d8fc0/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:26.841936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70b955-ac8e-4be6-b8f2-10e09dafea55 None admin] RBAC: Authorizing `identity:create_grant(project_id=309b82ea49ac4f668e791c60019886e8, user_id=2de08724c36744c1a919ef8c8c0d8fc0, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:26.843358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70b955-ac8e-4be6-b8f2-10e09dafea55 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:26.855806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 536/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:26:26 2022] PUT /identity/v3/projects/309b82ea49ac4f668e791c60019886e8/users/2de08724c36744c1a919ef8c8c0d8fc0/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:27.909608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f02b43-8ea4-4bc8-a1e7-79fd7b488ab6 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.910206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f02b43-8ea4-4bc8-a1e7-79fd7b488ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.910301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f02b43-8ea4-4bc8-a1e7-79fd7b488ab6 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.911726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 549/1085] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:27.916507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4e23-22c9-4dc2-8b10-13556710f606 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:27.916895 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4e23-22c9-4dc2-8b10-13556710f606 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:27.917172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4e23-22c9-4dc2-8b10-13556710f606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:27.934818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cecd4e23-22c9-4dc2-8b10-13556710f606 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:27.936024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cecd4e23-22c9-4dc2-8b10-13556710f606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:28.004395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 537/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:28.027572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f02b43-8ea4-4bc8-a1e7-79fd7b488ab6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.123208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:28.127524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f02b43-8ea4-4bc8-a1e7-79fd7b488ab6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.160253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47afa47-d846-41f4-81fb-b4516f7c4a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.160792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47afa47-d846-41f4-81fb-b4516f7c4a52 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.161068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47afa47-d846-41f4-81fb-b4516f7c4a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.161370 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47afa47-d846-41f4-81fb-b4516f7c4a52 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.171875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47afa47-d846-41f4-81fb-b4516f7c4a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f47afa47-d846-41f4-81fb-b4516f7c4a52 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:28.177016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:28.178023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:28.178914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 550/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:28.184500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd4e23-22c9-4dc2-8b10-13556710f606 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.290624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd4e23-22c9-4dc2-8b10-13556710f606 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.301602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:28.325708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.326306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.326578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.326839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.327483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:28.327774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.329105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.336688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 538/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:28.342310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47afa47-d846-41f4-81fb-b4516f7c4a52 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.439523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47afa47-d846-41f4-81fb-b4516f7c4a52 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.471496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.472110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.472906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.473181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.483580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.485011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:28.498964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:28.504889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:28.506124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:28.506124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:28.506124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:28.506124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 551/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:28.506366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.606421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7592f-702b-457b-b45d-58717b52b2bf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.640392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.640998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.641304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.641578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.642213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:28.642515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.643883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.655898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 539/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:28.661422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.676273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:28.759850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564ec21-d954-4594-aa5d-ef043e6fc51b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.792486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457da2a3-246e-41b1-8b3f-77cbbba186be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.793102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457da2a3-246e-41b1-8b3f-77cbbba186be None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.793406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457da2a3-246e-41b1-8b3f-77cbbba186be None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.793721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457da2a3-246e-41b1-8b3f-77cbbba186be None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.801507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457da2a3-246e-41b1-8b3f-77cbbba186be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.802933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457da2a3-246e-41b1-8b3f-77cbbba186be None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-457da2a3-246e-41b1-8b3f-77cbbba186be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:26:28.813521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:28.814631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:28.815575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 552/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:28.820932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.851134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:28.855428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e626b-98e3-4975-bf20-4098f2413b91 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.855714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e626b-98e3-4975-bf20-4098f2413b91 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.855987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e626b-98e3-4975-bf20-4098f2413b91 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.857484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 553/1092] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:26:28.862549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b450cd5-e2c8-44e4-8566-3e42b3955de8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.862831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b450cd5-e2c8-44e4-8566-3e42b3955de8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.863675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b450cd5-e2c8-44e4-8566-3e42b3955de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.884876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9b450cd5-e2c8-44e4-8566-3e42b3955de8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:28.886203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9b450cd5-e2c8-44e4-8566-3e42b3955de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:28.925181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0acd9-f337-40f3-8bb3-345bfc06ece9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:28.953176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 554/1093] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:28.961108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:28.961741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:28.962012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:28.962277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:28.962991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:28.963285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:28.964693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:28.975962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b450cd5-e2c8-44e4-8566-3e42b3955de8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:28.978112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:28.981878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 540/1094] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:28.987671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.073577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b450cd5-e2c8-44e4-8566-3e42b3955de8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.074093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.083949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd3f30-1dad-4fd2-8aa4-087da738bc75 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.108315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40893f28-4fa7-4c33-a1dc-59fec2857722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.108925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40893f28-4fa7-4c33-a1dc-59fec2857722 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.109199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40893f28-4fa7-4c33-a1dc-59fec2857722 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.109487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40893f28-4fa7-4c33-a1dc-59fec2857722 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.117221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab1eb5-d6ee-4e64-8c04-e0de24bde627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.117850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1eb5-d6ee-4e64-8c04-e0de24bde627 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.118126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1eb5-d6ee-4e64-8c04-e0de24bde627 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.118392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1eb5-d6ee-4e64-8c04-e0de24bde627 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.119343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40893f28-4fa7-4c33-a1dc-59fec2857722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-40893f28-4fa7-4c33-a1dc-59fec2857722 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:29.123906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:29.125025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:29.125910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 555/1095] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:29.131429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40893f28-4fa7-4c33-a1dc-59fec2857722 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.184953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab1eb5-d6ee-4e64-8c04-e0de24bde627 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.186427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab1eb5-d6ee-4e64-8c04-e0de24bde627 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.200274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.241709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40893f28-4fa7-4c33-a1dc-59fec2857722 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.282153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.282759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.283038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.283306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.283967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:29.284261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.285703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.294244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 556/1096] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:29.296932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.300103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.313536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 541/1097] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 13:26:29.314673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab1eb5-d6ee-4e64-8c04-e0de24bde627 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.399229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f93a3-2219-4b22-aad8-6ff634581b63 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.417065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab1eb5-d6ee-4e64-8c04-e0de24bde627 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.422050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.433541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e597dba-19d4-41c1-b844-04e6d41421d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.434161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e597dba-19d4-41c1-b844-04e6d41421d0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.434431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e597dba-19d4-41c1-b844-04e6d41421d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.434696 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e597dba-19d4-41c1-b844-04e6d41421d0 None admin] PATH_INFO: `/v3/users/2de08724c36744c1a919ef8c8c0d8fc0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.435821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e597dba-19d4-41c1-b844-04e6d41421d0 None admin] RBAC: Authorizing `identity:get_user(user_id=2de08724c36744c1a919ef8c8c0d8fc0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.437254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e597dba-19d4-41c1-b844-04e6d41421d0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.438830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 557/1098] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/users/2de08724c36744c1a919ef8c8c0d8fc0 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:29.444950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e597dba-19d4-41c1-b844-04e6d41421d0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.450463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.451090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.451364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.451633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.461949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.463383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:29.477446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:29.478462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:26:29.479371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:29.479371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:26:29.479371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:29.479371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 542/1099] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:26:29.484297 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.527028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.546466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e597dba-19d4-41c1-b844-04e6d41421d0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.585724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2976e4-8e75-4d4b-8484-c76fbc5dee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.586334 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2976e4-8e75-4d4b-8484-c76fbc5dee46 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.586612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2976e4-8e75-4d4b-8484-c76fbc5dee46 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.586883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2976e4-8e75-4d4b-8484-c76fbc5dee46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.586982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7d4cf-eff4-4ff3-8be1-91c50beef2a6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.620438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.621035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.621421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.621722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.622350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:29.622646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.624457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.637119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 543/1100] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:29.642757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.647260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.657307 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2976e4-8e75-4d4b-8484-c76fbc5dee46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.658716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2976e4-8e75-4d4b-8484-c76fbc5dee46 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.743273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968740ad-0aa4-4a28-b800-0621b6a39bb7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.744807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.757978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 558/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 13:26:29.763774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2976e4-8e75-4d4b-8484-c76fbc5dee46 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.785824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.786456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.786738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.787007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.793570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.794997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:26:29.806373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:29.807351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:29.808260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 544/1102] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:29.813463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.855988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:29.865934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2976e4-8e75-4d4b-8484-c76fbc5dee46 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.899534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe2c2f-1e0e-4e11-9718-f6b47c641ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.900152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2c2f-1e0e-4e11-9718-f6b47c641ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.900437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2c2f-1e0e-4e11-9718-f6b47c641ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.900706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2c2f-1e0e-4e11-9718-f6b47c641ff3 None admin] PATH_INFO: `/v3/projects/309b82ea49ac4f668e791c60019886e8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.901826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe2c2f-1e0e-4e11-9718-f6b47c641ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=309b82ea49ac4f668e791c60019886e8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.903256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe2c2f-1e0e-4e11-9718-f6b47c641ff3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.910556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 559/1103] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/projects/309b82ea49ac4f668e791c60019886e8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:29.912309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba74c10-52f4-4ae2-8a8c-85153f41cc90 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:29.914112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe2c2f-1e0e-4e11-9718-f6b47c641ff3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.945783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:29.946386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:29.946660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:29.946929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:29.947603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:29.947902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:29.949345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:29.961850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 545/1104] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:29.967928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:29.991877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:30.020757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe2c2f-1e0e-4e11-9718-f6b47c641ff3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:30.053602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab76b-2022-46c5-b245-4e28d08437f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:30.054202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab76b-2022-46c5-b245-4e28d08437f6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:30.054479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab76b-2022-46c5-b245-4e28d08437f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:30.054746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab76b-2022-46c5-b245-4e28d08437f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:30.055467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ab76b-2022-46c5-b245-4e28d08437f6 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=309b82ea49ac4f668e791c60019886e8, user.id=2de08724c36744c1a919ef8c8c0d8fc0) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:30.055766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ab76b-2022-46c5-b245-4e28d08437f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:30.057186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ab76b-2022-46c5-b245-4e28d08437f6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:30.063006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 560/1105] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/role_assignments?user.id=2de08724c36744c1a919ef8c8c0d8fc0&scope.project.id=309b82ea49ac4f668e791c60019886e8&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:30.070727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c371df2-9697-4b07-ae78-67d5352b041c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:30.110449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd2400-279a-42c0-ba6d-a93f8e9d92a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:30.111077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd2400-279a-42c0-ba6d-a93f8e9d92a4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:30.111352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd2400-279a-42c0-ba6d-a93f8e9d92a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:30.111619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd2400-279a-42c0-ba6d-a93f8e9d92a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:30.112358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd2400-279a-42c0-ba6d-a93f8e9d92a4 None admin] RBAC: Adding query filter params (role.id=e583cc6231c94d0a849b9dbe1a2677b3, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=4e395e059b5d436a8678bb302bcfafb5) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:30.112662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd2400-279a-42c0-ba6d-a93f8e9d92a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:30.114249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd2400-279a-42c0-ba6d-a93f8e9d92a4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:30.133270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 546/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:29 2022] GET /identity/v3/role_assignments?user.id=4e395e059b5d436a8678bb302bcfafb5&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=e583cc6231c94d0a849b9dbe1a2677b3 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:26:32.088343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71634d7-b1a9-43f7-baa4-c51e8ebe6160 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.088877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71634d7-b1a9-43f7-baa4-c51e8ebe6160 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.088877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71634d7-b1a9-43f7-baa4-c51e8ebe6160 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.090332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 561/1107] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:32.095268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f62c2b-96c1-47ab-a6e1-f11900229765 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.095665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f62c2b-96c1-47ab-a6e1-f11900229765 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.095801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f62c2b-96c1-47ab-a6e1-f11900229765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.114397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-68f62c2b-96c1-47ab-a6e1-f11900229765 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:32.115561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-68f62c2b-96c1-47ab-a6e1-f11900229765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:32.183206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 547/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:32.206450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71634d7-b1a9-43f7-baa4-c51e8ebe6160 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:32.230906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:32.305351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71634d7-b1a9-43f7-baa4-c51e8ebe6160 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:32.336779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20646b-b6cc-496b-90a3-8d7e5898302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.337411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20646b-b6cc-496b-90a3-8d7e5898302d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.337685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20646b-b6cc-496b-90a3-8d7e5898302d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.337947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20646b-b6cc-496b-90a3-8d7e5898302d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.339073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20646b-b6cc-496b-90a3-8d7e5898302d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.340437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20646b-b6cc-496b-90a3-8d7e5898302d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.341998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 562/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:32.348198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f62c2b-96c1-47ab-a6e1-f11900229765 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:32.365442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:32.443995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f62c2b-96c1-47ab-a6e1-f11900229765 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:32.476833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820aea54-0c33-4188-be61-f88521bacd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:32.477440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aea54-0c33-4188-be61-f88521bacd63 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:32.477713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aea54-0c33-4188-be61-f88521bacd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:32.477976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aea54-0c33-4188-be61-f88521bacd63 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:32.478600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820aea54-0c33-4188-be61-f88521bacd63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:32.479981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820aea54-0c33-4188-be61-f88521bacd63 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:32.501004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 548/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:26:32 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:34.510144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7befb-8ecb-43b0-bc78-c9ed86408d9f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.510745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7befb-8ecb-43b0-bc78-c9ed86408d9f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.510745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7befb-8ecb-43b0-bc78-c9ed86408d9f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.512016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 563/1111] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:34.517293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18098d-98df-4aeb-9618-4b813d66113c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.517723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18098d-98df-4aeb-9618-4b813d66113c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.517826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18098d-98df-4aeb-9618-4b813d66113c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.535857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9d18098d-98df-4aeb-9618-4b813d66113c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:34.536972 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9d18098d-98df-4aeb-9618-4b813d66113c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:34.600215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 549/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:34.623420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7befb-8ecb-43b0-bc78-c9ed86408d9f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.712495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:34.721838 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7befb-8ecb-43b0-bc78-c9ed86408d9f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.754885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd9e03-9737-40de-9014-01a158b93e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.755495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd9e03-9737-40de-9014-01a158b93e0b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.755769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd9e03-9737-40de-9014-01a158b93e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.756033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd9e03-9737-40de-9014-01a158b93e0b None admin] PATH_INFO: `/v3/roles/d0ccb065ab134c66b5033b67d0bb86b6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.757416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd9e03-9737-40de-9014-01a158b93e0b None admin] RBAC: Authorizing `identity:get_role(role_id=d0ccb065ab134c66b5033b67d0bb86b6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.758771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd9e03-9737-40de-9014-01a158b93e0b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.760064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 564/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:34 2022] GET /identity/v3/roles/d0ccb065ab134c66b5033b67d0bb86b6 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:34.765985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18098d-98df-4aeb-9618-4b813d66113c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:34.768152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:34.860780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18098d-98df-4aeb-9618-4b813d66113c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:34.893365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b057b3a-6a88-483f-b082-f80cef279203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:34.893972 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b057b3a-6a88-483f-b082-f80cef279203 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:34.894324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b057b3a-6a88-483f-b082-f80cef279203 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:34.894588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b057b3a-6a88-483f-b082-f80cef279203 None admin] PATH_INFO: `/v3/users/6b3dfd6007e6487b9ff87b8e4a34fcdf` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:34.907933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b057b3a-6a88-483f-b082-f80cef279203 None admin] RBAC: Authorizing `identity:get_user(user_id=6b3dfd6007e6487b9ff87b8e4a34fcdf)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:34.909380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b057b3a-6a88-483f-b082-f80cef279203 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:34.911067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 550/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:34 2022] GET /identity/v3/users/6b3dfd6007e6487b9ff87b8e4a34fcdf => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:34.916905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd9e03-9737-40de-9014-01a158b93e0b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:35.012112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd9e03-9737-40de-9014-01a158b93e0b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:35.043994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce89ee-802e-49d3-8f8f-a72606321432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:35.044580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce89ee-802e-49d3-8f8f-a72606321432 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:35.044856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce89ee-802e-49d3-8f8f-a72606321432 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:35.045125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce89ee-802e-49d3-8f8f-a72606321432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:35.071975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:35.111255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce89ee-802e-49d3-8f8f-a72606321432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:35.112623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce89ee-802e-49d3-8f8f-a72606321432 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:35.206190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 565/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:26:35.212116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b057b3a-6a88-483f-b082-f80cef279203 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:35.274107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:35.313715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b057b3a-6a88-483f-b082-f80cef279203 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:35.345421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d85dbd-701a-4c6c-9955-f13ea8ce2c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:35.346038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d85dbd-701a-4c6c-9955-f13ea8ce2c50 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:35.346310 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d85dbd-701a-4c6c-9955-f13ea8ce2c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:35.346576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d85dbd-701a-4c6c-9955-f13ea8ce2c50 None admin] PATH_INFO: `/v3/projects/309b82ea49ac4f668e791c60019886e8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:35.348208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d85dbd-701a-4c6c-9955-f13ea8ce2c50 None admin] RBAC: Authorizing `identity:get_project(project_id=309b82ea49ac4f668e791c60019886e8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:35.349635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d85dbd-701a-4c6c-9955-f13ea8ce2c50 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:35.351231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 551/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:35 2022] GET /identity/v3/projects/309b82ea49ac4f668e791c60019886e8 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:35.357270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce89ee-802e-49d3-8f8f-a72606321432 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:35.414400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:35.453785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce89ee-802e-49d3-8f8f-a72606321432 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:35.487529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8b2fb-0f30-4640-9524-7b00971d425b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:35.488111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b2fb-0f30-4640-9524-7b00971d425b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:35.488385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b2fb-0f30-4640-9524-7b00971d425b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:35.488650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b2fb-0f30-4640-9524-7b00971d425b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:35.489386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8b2fb-0f30-4640-9524-7b00971d425b None admin] RBAC: Adding query filter params (role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=309b82ea49ac4f668e791c60019886e8, user.id=6b3dfd6007e6487b9ff87b8e4a34fcdf) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:35.489685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8b2fb-0f30-4640-9524-7b00971d425b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:35.491090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8b2fb-0f30-4640-9524-7b00971d425b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:35.496201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 566/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:35 2022] GET /identity/v3/role_assignments?user.id=6b3dfd6007e6487b9ff87b8e4a34fcdf&scope.project.id=309b82ea49ac4f668e791c60019886e8&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:37.444761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b8fde-4f71-4bf2-9ff3-0a9b3c156989 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:37.445334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b8fde-4f71-4bf2-9ff3-0a9b3c156989 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:37.445334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b8fde-4f71-4bf2-9ff3-0a9b3c156989 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:37.446648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 552/1118] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:37.451738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4135-fa50-4d3c-b15b-95e673d79f68 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:37.452160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4135-fa50-4d3c-b15b-95e673d79f68 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:37.452281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4135-fa50-4d3c-b15b-95e673d79f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:37.470405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-89ed4135-fa50-4d3c-b15b-95e673d79f68 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:37.471576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-89ed4135-fa50-4d3c-b15b-95e673d79f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:37.535002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 567/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:37.556785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b8fde-4f71-4bf2-9ff3-0a9b3c156989 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:37.655574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b8fde-4f71-4bf2-9ff3-0a9b3c156989 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:37.681582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:37.689271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f7e57-bb09-45a4-8d8f-30f4fa993969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:37.689896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f7e57-bb09-45a4-8d8f-30f4fa993969 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:37.690175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f7e57-bb09-45a4-8d8f-30f4fa993969 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:37.690447 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f7e57-bb09-45a4-8d8f-30f4fa993969 None admin] PATH_INFO: `/v3/roles/d0ccb065ab134c66b5033b67d0bb86b6` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:37.691883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f7e57-bb09-45a4-8d8f-30f4fa993969 None admin] RBAC: Authorizing `identity:get_role(role_id=d0ccb065ab134c66b5033b67d0bb86b6)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:37.693265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f7e57-bb09-45a4-8d8f-30f4fa993969 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:37.694565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 553/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:37 2022] GET /identity/v3/roles/d0ccb065ab134c66b5033b67d0bb86b6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:37.700288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4135-fa50-4d3c-b15b-95e673d79f68 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:37.753946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:37.795931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4135-fa50-4d3c-b15b-95e673d79f68 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:37.828422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0918b0d-45ab-41cd-afef-a7ee015bb85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:37.829106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0918b0d-45ab-41cd-afef-a7ee015bb85f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:37.829405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0918b0d-45ab-41cd-afef-a7ee015bb85f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:37.829669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0918b0d-45ab-41cd-afef-a7ee015bb85f None admin] PATH_INFO: `/v3/users/6b3dfd6007e6487b9ff87b8e4a34fcdf` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:37.830755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0918b0d-45ab-41cd-afef-a7ee015bb85f None admin] RBAC: Authorizing `identity:get_user(user_id=6b3dfd6007e6487b9ff87b8e4a34fcdf)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:37.832155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0918b0d-45ab-41cd-afef-a7ee015bb85f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:37.833731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 568/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:37 2022] GET /identity/v3/users/6b3dfd6007e6487b9ff87b8e4a34fcdf => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:37.839317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f7e57-bb09-45a4-8d8f-30f4fa993969 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:37.935289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f7e57-bb09-45a4-8d8f-30f4fa993969 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:37.967750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54562b-9852-46d8-802c-15bfe041e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:37.968370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54562b-9852-46d8-802c-15bfe041e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:37.968644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54562b-9852-46d8-802c-15bfe041e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:37.968912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54562b-9852-46d8-802c-15bfe041e3d2 None admin] PATH_INFO: `/v3/projects/309b82ea49ac4f668e791c60019886e8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:37.970035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54562b-9852-46d8-802c-15bfe041e3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=309b82ea49ac4f668e791c60019886e8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:37.971456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54562b-9852-46d8-802c-15bfe041e3d2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:37.973093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 554/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:37 2022] GET /identity/v3/projects/309b82ea49ac4f668e791c60019886e8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:37.979041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0918b0d-45ab-41cd-afef-a7ee015bb85f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:38.075893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0918b0d-45ab-41cd-afef-a7ee015bb85f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:38.097733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:38.109636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e87474-aba2-4fc3-9efd-87b1d2a5d566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:38.110281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e87474-aba2-4fc3-9efd-87b1d2a5d566 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:38.110561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e87474-aba2-4fc3-9efd-87b1d2a5d566 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:38.110829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e87474-aba2-4fc3-9efd-87b1d2a5d566 None admin] PATH_INFO: `/v3/projects/309b82ea49ac4f668e791c60019886e8/users/6b3dfd6007e6487b9ff87b8e4a34fcdf/roles/d0ccb065ab134c66b5033b67d0bb86b6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:38.113495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e87474-aba2-4fc3-9efd-87b1d2a5d566 None admin] RBAC: Authorizing `identity:create_grant(project_id=309b82ea49ac4f668e791c60019886e8, user_id=6b3dfd6007e6487b9ff87b8e4a34fcdf, role_id=d0ccb065ab134c66b5033b67d0bb86b6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:38.114916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e87474-aba2-4fc3-9efd-87b1d2a5d566 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:38.126559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 569/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:26:37 2022] PUT /identity/v3/projects/309b82ea49ac4f668e791c60019886e8/users/6b3dfd6007e6487b9ff87b8e4a34fcdf/roles/d0ccb065ab134c66b5033b67d0bb86b6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:40.093210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080edcc-8ac1-49ef-a144-1c0bf940178d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.093764 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080edcc-8ac1-49ef-a144-1c0bf940178d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.093764 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080edcc-8ac1-49ef-a144-1c0bf940178d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.095091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 555/1124] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:40.100805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b8a3b-a100-46b0-b014-1116589c263e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.101250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b8a3b-a100-46b0-b014-1116589c263e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.101357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b8a3b-a100-46b0-b014-1116589c263e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.118818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-400b8a3b-a100-46b0-b014-1116589c263e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:40.119974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-400b8a3b-a100-46b0-b014-1116589c263e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:40.182392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 570/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:40.205304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080edcc-8ac1-49ef-a144-1c0bf940178d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:40.267435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:40.306264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080edcc-8ac1-49ef-a144-1c0bf940178d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:40.338574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a1f57-572e-4656-99d0-a7bfd7ccf821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:40.339187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1f57-572e-4656-99d0-a7bfd7ccf821 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.339466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1f57-572e-4656-99d0-a7bfd7ccf821 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.339735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1f57-572e-4656-99d0-a7bfd7ccf821 None admin] PATH_INFO: `/v3/roles/d0ccb065ab134c66b5033b67d0bb86b6` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.341104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a1f57-572e-4656-99d0-a7bfd7ccf821 None admin] RBAC: Authorizing `identity:get_role(role_id=d0ccb065ab134c66b5033b67d0bb86b6)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:40.342501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a1f57-572e-4656-99d0-a7bfd7ccf821 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:40.343848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 556/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:40 2022] GET /identity/v3/roles/d0ccb065ab134c66b5033b67d0bb86b6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:40.350295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b8a3b-a100-46b0-b014-1116589c263e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:40.444695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b8a3b-a100-46b0-b014-1116589c263e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:40.459530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:40.480769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f377ca4-711b-44e0-86a2-e70e689eb402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:40.481405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f377ca4-711b-44e0-86a2-e70e689eb402 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.481679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f377ca4-711b-44e0-86a2-e70e689eb402 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.481949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f377ca4-711b-44e0-86a2-e70e689eb402 None admin] PATH_INFO: `/v3/users/6b3dfd6007e6487b9ff87b8e4a34fcdf` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.483278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f377ca4-711b-44e0-86a2-e70e689eb402 None admin] RBAC: Authorizing `identity:get_user(user_id=6b3dfd6007e6487b9ff87b8e4a34fcdf)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:40.484689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f377ca4-711b-44e0-86a2-e70e689eb402 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:40.486395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 571/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:40 2022] GET /identity/v3/users/6b3dfd6007e6487b9ff87b8e4a34fcdf => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:40.492726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a1f57-572e-4656-99d0-a7bfd7ccf821 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:40.586738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a1f57-572e-4656-99d0-a7bfd7ccf821 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:40.615178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:40.620212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92986da-2e56-4e82-a97b-d79a9cbb9aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:40.620805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92986da-2e56-4e82-a97b-d79a9cbb9aed None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.621079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92986da-2e56-4e82-a97b-d79a9cbb9aed None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.621380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92986da-2e56-4e82-a97b-d79a9cbb9aed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.685765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92986da-2e56-4e82-a97b-d79a9cbb9aed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:40.687189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92986da-2e56-4e82-a97b-d79a9cbb9aed None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:40.779899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 557/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:26:40.786442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f377ca4-711b-44e0-86a2-e70e689eb402 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:40.879580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f377ca4-711b-44e0-86a2-e70e689eb402 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:40.912799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e15cf7-dabc-42cb-a517-386c6447d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:40.913464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e15cf7-dabc-42cb-a517-386c6447d9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:40.913738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e15cf7-dabc-42cb-a517-386c6447d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:40.914005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e15cf7-dabc-42cb-a517-386c6447d9e2 None admin] PATH_INFO: `/v3/projects/309b82ea49ac4f668e791c60019886e8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:40.915093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e15cf7-dabc-42cb-a517-386c6447d9e2 None admin] RBAC: Authorizing `identity:get_project(project_id=309b82ea49ac4f668e791c60019886e8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:40.916539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e15cf7-dabc-42cb-a517-386c6447d9e2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:40.918171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 572/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:40 2022] GET /identity/v3/projects/309b82ea49ac4f668e791c60019886e8 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:40.924112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92986da-2e56-4e82-a97b-d79a9cbb9aed None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:40.957720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:41.023642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92986da-2e56-4e82-a97b-d79a9cbb9aed None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:41.056342 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73737454-0933-44f7-a732-c33f29bba7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:41.056934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73737454-0933-44f7-a732-c33f29bba7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:41.057209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73737454-0933-44f7-a732-c33f29bba7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:41.057488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73737454-0933-44f7-a732-c33f29bba7f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:41.058198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73737454-0933-44f7-a732-c33f29bba7f9 None admin] RBAC: Adding query filter params (role.id=d0ccb065ab134c66b5033b67d0bb86b6, scope.project.id=309b82ea49ac4f668e791c60019886e8, user.id=6b3dfd6007e6487b9ff87b8e4a34fcdf) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:41.058498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73737454-0933-44f7-a732-c33f29bba7f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:41.059900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73737454-0933-44f7-a732-c33f29bba7f9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:41.065440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 558/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:40 2022] GET /identity/v3/role_assignments?user.id=6b3dfd6007e6487b9ff87b8e4a34fcdf&scope.project.id=309b82ea49ac4f668e791c60019886e8&role.id=d0ccb065ab134c66b5033b67d0bb86b6 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:43.054682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e59c7-fa79-407f-9748-0cdc7d6d9d7c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.055322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e59c7-fa79-407f-9748-0cdc7d6d9d7c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.055322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e59c7-fa79-407f-9748-0cdc7d6d9d7c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.056546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 573/1131] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:43.061579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221a62c-7193-4aa2-8875-acbf2a2a9fa8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.061960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221a62c-7193-4aa2-8875-acbf2a2a9fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.062124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221a62c-7193-4aa2-8875-acbf2a2a9fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.080093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b221a62c-7193-4aa2-8875-acbf2a2a9fa8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:43.081349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b221a62c-7193-4aa2-8875-acbf2a2a9fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:43.144987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 559/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:43.167564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e59c7-fa79-407f-9748-0cdc7d6d9d7c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.216204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:43.266425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e59c7-fa79-407f-9748-0cdc7d6d9d7c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.299170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404419b-4594-45d2-864a-f9776a5957f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.299776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404419b-4594-45d2-864a-f9776a5957f8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.300578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404419b-4594-45d2-864a-f9776a5957f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.300853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404419b-4594-45d2-864a-f9776a5957f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.301983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404419b-4594-45d2-864a-f9776a5957f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.303396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404419b-4594-45d2-864a-f9776a5957f8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.304982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 574/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:43.310765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221a62c-7193-4aa2-8875-acbf2a2a9fa8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:43.337287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:43.408393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221a62c-7193-4aa2-8875-acbf2a2a9fa8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:43.441486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a4d6e-51a6-4373-a6b4-d4924fb930f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:43.442084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a4d6e-51a6-4373-a6b4-d4924fb930f2 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:43.442360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a4d6e-51a6-4373-a6b4-d4924fb930f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:43.442628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a4d6e-51a6-4373-a6b4-d4924fb930f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:43.443268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a4d6e-51a6-4373-a6b4-d4924fb930f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:43.444703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a4d6e-51a6-4373-a6b4-d4924fb930f2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:43.470174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 560/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:26:43 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:45.509793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59a2df-9cc8-4bb0-a74b-3d99b01dfa7d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.510315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59a2df-9cc8-4bb0-a74b-3d99b01dfa7d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.510315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59a2df-9cc8-4bb0-a74b-3d99b01dfa7d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.511650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 575/1135] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:45.516649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25aedc-10f1-44ed-ae33-8e394f232369 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.517080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25aedc-10f1-44ed-ae33-8e394f232369 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.517188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25aedc-10f1-44ed-ae33-8e394f232369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.534869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fa25aedc-10f1-44ed-ae33-8e394f232369 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:45.535973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fa25aedc-10f1-44ed-ae33-8e394f232369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:45.600006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 561/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:45.624231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59a2df-9cc8-4bb0-a74b-3d99b01dfa7d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:45.720608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59a2df-9cc8-4bb0-a74b-3d99b01dfa7d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:45.736795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:45.754415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fabc4-593a-4b8a-bb46-a737dc5d99f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:45.755023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fabc4-593a-4b8a-bb46-a737dc5d99f5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.755299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fabc4-593a-4b8a-bb46-a737dc5d99f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.755566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fabc4-593a-4b8a-bb46-a737dc5d99f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.756659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fabc4-593a-4b8a-bb46-a737dc5d99f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:45.758104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fabc4-593a-4b8a-bb46-a737dc5d99f5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:45.759640 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 576/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:26:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:45.765198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25aedc-10f1-44ed-ae33-8e394f232369 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:45.775739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:45.861785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25aedc-10f1-44ed-ae33-8e394f232369 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:45.893804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc9598-2f07-4320-b308-29ded9476a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:45.894400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc9598-2f07-4320-b308-29ded9476a91 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:45.894675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc9598-2f07-4320-b308-29ded9476a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:45.894943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc9598-2f07-4320-b308-29ded9476a91 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:45.895578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc9598-2f07-4320-b308-29ded9476a91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:45.896984 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc9598-2f07-4320-b308-29ded9476a91 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:45.917608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 562/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:26:45 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:26:47.907071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c704a65-582f-4e23-859e-1f5ee7159e21 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:47.907621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c704a65-582f-4e23-859e-1f5ee7159e21 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:47.907621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c704a65-582f-4e23-859e-1f5ee7159e21 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:47.908934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 577/1139] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:47.914050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386fea-0c62-4ed2-a0cb-b63e9539b4c2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:47.914434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386fea-0c62-4ed2-a0cb-b63e9539b4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:47.914600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386fea-0c62-4ed2-a0cb-b63e9539b4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:47.932032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f5386fea-0c62-4ed2-a0cb-b63e9539b4c2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:47.933200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f5386fea-0c62-4ed2-a0cb-b63e9539b4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:47.989073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:47.997935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 563/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:48.021716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c704a65-582f-4e23-859e-1f5ee7159e21 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:48.118029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c704a65-582f-4e23-859e-1f5ee7159e21 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:48.150660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf833714-ad26-4402-98f6-c220a83b622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:48.151260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf833714-ad26-4402-98f6-c220a83b622b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:48.151533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf833714-ad26-4402-98f6-c220a83b622b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:48.151797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf833714-ad26-4402-98f6-c220a83b622b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:48.161496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf833714-ad26-4402-98f6-c220a83b622b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-cf833714-ad26-4402-98f6-c220a83b622b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:48.165973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:48.167103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:48.168138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 578/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:48.172975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5386fea-0c62-4ed2-a0cb-b63e9539b4c2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:48.268612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5386fea-0c62-4ed2-a0cb-b63e9539b4c2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:48.301144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:48.301774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:48.302049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:48.302344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:48.302995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:48.303290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:48.304641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:48.312260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 564/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:48.318877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf833714-ad26-4402-98f6-c220a83b622b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:48.363204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:48.414807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf833714-ad26-4402-98f6-c220a83b622b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:48.448684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f92244-a49e-4441-9524-6157a820257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:48.449315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f92244-a49e-4441-9524-6157a820257b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:48.449591 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f92244-a49e-4441-9524-6157a820257b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:48.449857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f92244-a49e-4441-9524-6157a820257b None admin] PATH_INFO: `/v3/users/4ef4dbd6613b4f16bca6687fa182bf7f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:48.462227 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f92244-a49e-4441-9524-6157a820257b None admin] RBAC: Authorizing `identity:get_user(user_id=4ef4dbd6613b4f16bca6687fa182bf7f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:48.463663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f92244-a49e-4441-9524-6157a820257b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:48.465280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 579/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:48 2022] GET /identity/v3/users/4ef4dbd6613b4f16bca6687fa182bf7f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:48.470962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:48.525117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:48.568179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b36da-93b7-4b56-8fdc-69601067107a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:48.600212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07354f76-d26d-472b-914b-26950c07afc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:48.600801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07354f76-d26d-472b-914b-26950c07afc4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:48.601074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07354f76-d26d-472b-914b-26950c07afc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:48.601369 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07354f76-d26d-472b-914b-26950c07afc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:48.665553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07354f76-d26d-472b-914b-26950c07afc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:48.666907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07354f76-d26d-472b-914b-26950c07afc4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:48.717664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:48.760773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 565/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:26:48.766633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f92244-a49e-4441-9524-6157a820257b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:48.861095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f92244-a49e-4441-9524-6157a820257b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:48.862643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:48.895550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22059c1f-d992-46d3-9c6d-f84d4a9bf37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:48.896161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22059c1f-d992-46d3-9c6d-f84d4a9bf37e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:48.896433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22059c1f-d992-46d3-9c6d-f84d4a9bf37e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:48.896708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22059c1f-d992-46d3-9c6d-f84d4a9bf37e None admin] PATH_INFO: `/v3/projects/ebfae213605d4221af7483aacfcb1d05` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:48.897810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22059c1f-d992-46d3-9c6d-f84d4a9bf37e None admin] RBAC: Authorizing `identity:get_project(project_id=ebfae213605d4221af7483aacfcb1d05)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:48.899225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22059c1f-d992-46d3-9c6d-f84d4a9bf37e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:48.900805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 580/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:48 2022] GET /identity/v3/projects/ebfae213605d4221af7483aacfcb1d05 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:48.906791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07354f76-d26d-472b-914b-26950c07afc4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:49.002147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07354f76-d26d-472b-914b-26950c07afc4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:49.034307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a3cb7-3aa1-4157-8634-bc006f4ca741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:49.034910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a3cb7-3aa1-4157-8634-bc006f4ca741 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:49.035179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a3cb7-3aa1-4157-8634-bc006f4ca741 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:49.035449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a3cb7-3aa1-4157-8634-bc006f4ca741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:49.036149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a3cb7-3aa1-4157-8634-bc006f4ca741 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=ebfae213605d4221af7483aacfcb1d05, user.id=4ef4dbd6613b4f16bca6687fa182bf7f) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:49.036454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a3cb7-3aa1-4157-8634-bc006f4ca741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:49.037872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a3cb7-3aa1-4157-8634-bc006f4ca741 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:49.042919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 566/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:48 2022] GET /identity/v3/role_assignments?user.id=4ef4dbd6613b4f16bca6687fa182bf7f&scope.project.id=ebfae213605d4221af7483aacfcb1d05&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:50.999369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08915215-b5d7-469e-a5c0-2eb97b06b85e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:50.999901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08915215-b5d7-469e-a5c0-2eb97b06b85e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:50.999901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08915215-b5d7-469e-a5c0-2eb97b06b85e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.001248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 581/1147] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:51.006911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e542c-d7d5-4812-9cdc-21ce1cd3ef10 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.007344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e542c-d7d5-4812-9cdc-21ce1cd3ef10 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.007448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e542c-d7d5-4812-9cdc-21ce1cd3ef10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.025366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-636e542c-d7d5-4812-9cdc-21ce1cd3ef10 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:51.026529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-636e542c-d7d5-4812-9cdc-21ce1cd3ef10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:51.039979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:51.090517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 567/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:51.112584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08915215-b5d7-469e-a5c0-2eb97b06b85e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.209100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08915215-b5d7-469e-a5c0-2eb97b06b85e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.241290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468815a7-1647-4f77-af6f-038a241e16e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.241890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468815a7-1647-4f77-af6f-038a241e16e2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.242160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468815a7-1647-4f77-af6f-038a241e16e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.242428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468815a7-1647-4f77-af6f-038a241e16e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.252172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468815a7-1647-4f77-af6f-038a241e16e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-468815a7-1647-4f77-af6f-038a241e16e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:51.257535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:51.258626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:51.259672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 582/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:26:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:51.264586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e542c-d7d5-4812-9cdc-21ce1cd3ef10 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.359973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e542c-d7d5-4812-9cdc-21ce1cd3ef10 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.391515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.392109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.392386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.392655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.393327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:51.394153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.395490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.403108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 568/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:26:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:51.408933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468815a7-1647-4f77-af6f-038a241e16e2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.438838 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:51.506082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468815a7-1647-4f77-af6f-038a241e16e2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.538656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e95d9-a178-4edf-8b34-9d458800c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.539256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e95d9-a178-4edf-8b34-9d458800c6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.539528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e95d9-a178-4edf-8b34-9d458800c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.539791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e95d9-a178-4edf-8b34-9d458800c6f7 None admin] PATH_INFO: `/v3/users/4ef4dbd6613b4f16bca6687fa182bf7f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.540889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e95d9-a178-4edf-8b34-9d458800c6f7 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef4dbd6613b4f16bca6687fa182bf7f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.542354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e95d9-a178-4edf-8b34-9d458800c6f7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.543885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 583/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:51 2022] GET /identity/v3/users/4ef4dbd6613b4f16bca6687fa182bf7f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:51.549588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.564322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:51.645806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9ed67-ba8d-4dc8-b467-87f0fc865f09 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.676992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636c40d-5d48-4244-a848-caa185488e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.677627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636c40d-5d48-4244-a848-caa185488e42 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.677909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636c40d-5d48-4244-a848-caa185488e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.678178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636c40d-5d48-4244-a848-caa185488e42 None admin] PATH_INFO: `/v3/projects/ebfae213605d4221af7483aacfcb1d05` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.679267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636c40d-5d48-4244-a848-caa185488e42 None admin] RBAC: Authorizing `identity:get_project(project_id=ebfae213605d4221af7483aacfcb1d05)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.680672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636c40d-5d48-4244-a848-caa185488e42 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.682287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 569/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:51 2022] GET /identity/v3/projects/ebfae213605d4221af7483aacfcb1d05 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:51.688056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e95d9-a178-4edf-8b34-9d458800c6f7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:51.780340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:51.783816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e95d9-a178-4edf-8b34-9d458800c6f7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:51.817661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d4012-61eb-42b5-92fc-f27c884eea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:51.818300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d4012-61eb-42b5-92fc-f27c884eea94 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:51.818576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d4012-61eb-42b5-92fc-f27c884eea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:51.818846 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d4012-61eb-42b5-92fc-f27c884eea94 None admin] PATH_INFO: `/v3/projects/ebfae213605d4221af7483aacfcb1d05/users/4ef4dbd6613b4f16bca6687fa182bf7f/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:51.820703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d4012-61eb-42b5-92fc-f27c884eea94 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebfae213605d4221af7483aacfcb1d05, user_id=4ef4dbd6613b4f16bca6687fa182bf7f, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:51.822151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d4012-61eb-42b5-92fc-f27c884eea94 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:51.835120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 584/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:26:51 2022] PUT /identity/v3/projects/ebfae213605d4221af7483aacfcb1d05/users/4ef4dbd6613b4f16bca6687fa182bf7f/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:26:53.791392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f651f-64ab-4c8c-93b2-6df4b9d249a0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.791952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f651f-64ab-4c8c-93b2-6df4b9d249a0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.791952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f651f-64ab-4c8c-93b2-6df4b9d249a0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.793278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 570/1154] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:53.798301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29836a2c-a9c5-4ad8-990a-253b94a60c40 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:53.798738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29836a2c-a9c5-4ad8-990a-253b94a60c40 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:53.798832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29836a2c-a9c5-4ad8-990a-253b94a60c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:53.817345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-29836a2c-a9c5-4ad8-990a-253b94a60c40 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:53.818514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-29836a2c-a9c5-4ad8-990a-253b94a60c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:53.882433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 585/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:53.904987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f651f-64ab-4c8c-93b2-6df4b9d249a0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:53.984963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:54.004878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f651f-64ab-4c8c-93b2-6df4b9d249a0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.040729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926a197-8c7a-48f2-95a9-ab8e2ed04734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.041375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926a197-8c7a-48f2-95a9-ab8e2ed04734 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.041700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926a197-8c7a-48f2-95a9-ab8e2ed04734 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.041919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926a197-8c7a-48f2-95a9-ab8e2ed04734 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.052097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926a197-8c7a-48f2-95a9-ab8e2ed04734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-8926a197-8c7a-48f2-95a9-ab8e2ed04734 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:26:54.056703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:54.057835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:26:54.058759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 571/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:26:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:54.063878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29836a2c-a9c5-4ad8-990a-253b94a60c40 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.146263 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:54.158342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29836a2c-a9c5-4ad8-990a-253b94a60c40 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.191122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.191705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.191981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.192248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.192898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:54.193191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.194566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.202177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 586/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:26:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:54.208252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926a197-8c7a-48f2-95a9-ab8e2ed04734 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.304328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926a197-8c7a-48f2-95a9-ab8e2ed04734 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.336553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29605e-cc7a-4fb8-9faf-458503fe6824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.337163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29605e-cc7a-4fb8-9faf-458503fe6824 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.337465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29605e-cc7a-4fb8-9faf-458503fe6824 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.337731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29605e-cc7a-4fb8-9faf-458503fe6824 None admin] PATH_INFO: `/v3/users/4ef4dbd6613b4f16bca6687fa182bf7f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.338814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29605e-cc7a-4fb8-9faf-458503fe6824 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef4dbd6613b4f16bca6687fa182bf7f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.340213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29605e-cc7a-4fb8-9faf-458503fe6824 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.341780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 572/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:26:54 2022] GET /identity/v3/users/4ef4dbd6613b4f16bca6687fa182bf7f => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:54.348016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.443256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5a474-20cc-46a6-b533-850555053346 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.475797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71074f0-8484-4541-8bef-a184fdf67217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.476383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71074f0-8484-4541-8bef-a184fdf67217 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.476654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71074f0-8484-4541-8bef-a184fdf67217 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.476917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71074f0-8484-4541-8bef-a184fdf67217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.495127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:54.542108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71074f0-8484-4541-8bef-a184fdf67217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.543552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71074f0-8484-4541-8bef-a184fdf67217 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.635446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 587/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:26:54.641284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29605e-cc7a-4fb8-9faf-458503fe6824 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.652620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:54.736460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29605e-cc7a-4fb8-9faf-458503fe6824 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.768121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8884e70-73e6-4bb6-a678-8d1fd7564939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.769354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884e70-73e6-4bb6-a678-8d1fd7564939 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.769627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884e70-73e6-4bb6-a678-8d1fd7564939 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.769893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884e70-73e6-4bb6-a678-8d1fd7564939 None admin] PATH_INFO: `/v3/projects/ebfae213605d4221af7483aacfcb1d05` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.771079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8884e70-73e6-4bb6-a678-8d1fd7564939 None admin] RBAC: Authorizing `identity:get_project(project_id=ebfae213605d4221af7483aacfcb1d05)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.772633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8884e70-73e6-4bb6-a678-8d1fd7564939 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.774354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 573/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:26:54 2022] GET /identity/v3/projects/ebfae213605d4221af7483aacfcb1d05 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:54.780488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71074f0-8484-4541-8bef-a184fdf67217 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:54.827436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:54.875158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71074f0-8484-4541-8bef-a184fdf67217 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:54.907874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9848960-de04-4dd0-ae22-c1520514897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:54.908396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848960-de04-4dd0-ae22-c1520514897f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:54.908670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848960-de04-4dd0-ae22-c1520514897f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:54.908934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848960-de04-4dd0-ae22-c1520514897f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:54.909697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9848960-de04-4dd0-ae22-c1520514897f None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=ebfae213605d4221af7483aacfcb1d05, user.id=4ef4dbd6613b4f16bca6687fa182bf7f) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:54.909995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9848960-de04-4dd0-ae22-c1520514897f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:54.911405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9848960-de04-4dd0-ae22-c1520514897f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:54.916876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 588/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:26:54 2022] GET /identity/v3/role_assignments?user.id=4ef4dbd6613b4f16bca6687fa182bf7f&scope.project.id=ebfae213605d4221af7483aacfcb1d05&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:56.891773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dd095-d7cb-4284-ab6f-d7ceb32f030b None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:56.892300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dd095-d7cb-4284-ab6f-d7ceb32f030b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:56.892300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dd095-d7cb-4284-ab6f-d7ceb32f030b None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:56.893728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 574/1162] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:56.898744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73578-876e-41cb-8b41-91f323de1b60 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:56.899137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73578-876e-41cb-8b41-91f323de1b60 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:56.899285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73578-876e-41cb-8b41-91f323de1b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:56.917247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f3a73578-876e-41cb-8b41-91f323de1b60 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:56.918423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f3a73578-876e-41cb-8b41-91f323de1b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:56.981349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 589/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:26:57.003301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dd095-d7cb-4284-ab6f-d7ceb32f030b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:57.077701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:57.101077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dd095-d7cb-4284-ab6f-d7ceb32f030b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:57.133532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2136d-d726-4d8a-a725-e1bfc67f02f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:57.134115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2136d-d726-4d8a-a725-e1bfc67f02f2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:57.134385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2136d-d726-4d8a-a725-e1bfc67f02f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:57.134645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2136d-d726-4d8a-a725-e1bfc67f02f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:57.198531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2136d-d726-4d8a-a725-e1bfc67f02f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:57.199891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2136d-d726-4d8a-a725-e1bfc67f02f2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:57.273456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:57.296394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 575/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:26:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:26:57.302225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a73578-876e-41cb-8b41-91f323de1b60 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:57.346589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:57.399187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a73578-876e-41cb-8b41-91f323de1b60 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:57.432299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdc59a-b8f3-49ec-ace5-a10672189fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:57.432911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdc59a-b8f3-49ec-ace5-a10672189fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:57.433184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdc59a-b8f3-49ec-ace5-a10672189fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:57.433480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdc59a-b8f3-49ec-ace5-a10672189fe7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:57.439845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdc59a-b8f3-49ec-ace5-a10672189fe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:57.441333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdc59a-b8f3-49ec-ace5-a10672189fe7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bccdc59a-b8f3-49ec-ace5-a10672189fe7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:26:57.452437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:26:57.453679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:26:57.454562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 590/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:26:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:26:57.459960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2136d-d726-4d8a-a725-e1bfc67f02f2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:57.555909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2136d-d726-4d8a-a725-e1bfc67f02f2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:57.588833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bccfe-cbb7-414c-8ee3-3640ffae6d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:57.589473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bccfe-cbb7-414c-8ee3-3640ffae6d9d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:57.589751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bccfe-cbb7-414c-8ee3-3640ffae6d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:57.590018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bccfe-cbb7-414c-8ee3-3640ffae6d9d None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:57.590649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bccfe-cbb7-414c-8ee3-3640ffae6d9d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:26:57.590945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bccfe-cbb7-414c-8ee3-3640ffae6d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:57.592290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bccfe-cbb7-414c-8ee3-3640ffae6d9d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:57.599732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 576/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 10 13:26:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:26:59.578721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2aaaa-86b0-4553-9f27-cf6b783cd58a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:59.579251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2aaaa-86b0-4553-9f27-cf6b783cd58a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:59.579251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2aaaa-86b0-4553-9f27-cf6b783cd58a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:59.580530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 591/1167] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:26:59.586287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe387e6-4bd4-4790-9b09-c073bd05e819 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:59.586712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe387e6-4bd4-4790-9b09-c073bd05e819 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:59.586816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe387e6-4bd4-4790-9b09-c073bd05e819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:59.604861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bbe387e6-4bd4-4790-9b09-c073bd05e819 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:59.606074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bbe387e6-4bd4-4790-9b09-c073bd05e819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:59.638407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:59.670951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 577/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:26:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:26:59.693042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2aaaa-86b0-4553-9f27-cf6b783cd58a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:59.795244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2aaaa-86b0-4553-9f27-cf6b783cd58a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:59.797551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:59.832957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716499c7-72e7-43f9-a244-51df0e49682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:59.833502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716499c7-72e7-43f9-a244-51df0e49682e None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:59.833777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716499c7-72e7-43f9-a244-51df0e49682e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:59.834042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716499c7-72e7-43f9-a244-51df0e49682e None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:59.834684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716499c7-72e7-43f9-a244-51df0e49682e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:59.836032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716499c7-72e7-43f9-a244-51df0e49682e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:59.851065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 592/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Aug 10 13:26:59 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:27:01.846060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258d184-316d-41b2-9a7e-364373125e67 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.846617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258d184-316d-41b2-9a7e-364373125e67 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.846712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258d184-316d-41b2-9a7e-364373125e67 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.848030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 578/1170] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:27:01.853150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6defb-9cbb-40ac-8bbb-22e65cdffe18 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.853610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6defb-9cbb-40ac-8bbb-22e65cdffe18 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.853726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6defb-9cbb-40ac-8bbb-22e65cdffe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.871830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-afd6defb-9cbb-40ac-8bbb-22e65cdffe18 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:01.872983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-afd6defb-9cbb-40ac-8bbb-22e65cdffe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.936621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 593/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:27:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:27:01.959446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258d184-316d-41b2-9a7e-364373125e67 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:02.056383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258d184-316d-41b2-9a7e-364373125e67 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.088490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d89964-343c-45bc-847f-3aed63b97ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.089100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d89964-343c-45bc-847f-3aed63b97ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.089403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d89964-343c-45bc-847f-3aed63b97ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.089672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d89964-343c-45bc-847f-3aed63b97ff8 None admin] PATH_INFO: `/v3/domains/0d9185d8fc844a909ba842c2dc0e97c0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.100067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d89964-343c-45bc-847f-3aed63b97ff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0d9185d8fc844a909ba842c2dc0e97c0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.101455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d89964-343c-45bc-847f-3aed63b97ff8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.102995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 579/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Aug 10 13:27:01 2022] GET /identity/v3/domains/0d9185d8fc844a909ba842c2dc0e97c0 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:02.108823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6defb-9cbb-40ac-8bbb-22e65cdffe18 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:02.195526 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:02.202242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6defb-9cbb-40ac-8bbb-22e65cdffe18 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:02.234848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa1793-65a9-46d6-8f6b-ad2a1417c837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:02.235435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1793-65a9-46d6-8f6b-ad2a1417c837 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:02.235708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1793-65a9-46d6-8f6b-ad2a1417c837 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:02.235973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1793-65a9-46d6-8f6b-ad2a1417c837 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:02.236602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa1793-65a9-46d6-8f6b-ad2a1417c837 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:02.238675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa1793-65a9-46d6-8f6b-ad2a1417c837 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:02.263776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 594/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:27:02 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:27:04.242429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ee207-053e-4b81-8ae5-816386999076 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:04.243068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ee207-053e-4b81-8ae5-816386999076 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:04.243068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ee207-053e-4b81-8ae5-816386999076 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:04.244294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 580/1174] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:27:04.249312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc8c6-b863-4b89-a8ef-a9a5ecad6532 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:04.249777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc8c6-b863-4b89-a8ef-a9a5ecad6532 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:04.249876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc8c6-b863-4b89-a8ef-a9a5ecad6532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:04.267253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-63bcc8c6-b863-4b89-a8ef-a9a5ecad6532 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:04.268932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-63bcc8c6-b863-4b89-a8ef-a9a5ecad6532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:04.332435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 595/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:27:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:27:04.356311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ee207-053e-4b81-8ae5-816386999076 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:04.386519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:04.455626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ee207-053e-4b81-8ae5-816386999076 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:04.488856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f2da6-bd9c-4d9f-b352-b0e0ee0d170f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:04.489489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2da6-bd9c-4d9f-b352-b0e0ee0d170f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:04.489766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2da6-bd9c-4d9f-b352-b0e0ee0d170f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:04.490037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2da6-bd9c-4d9f-b352-b0e0ee0d170f None admin] PATH_INFO: `/v3/domains/0d9185d8fc844a909ba842c2dc0e97c0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:04.491119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f2da6-bd9c-4d9f-b352-b0e0ee0d170f None admin] RBAC: Authorizing `identity:get_domain(domain_id=0d9185d8fc844a909ba842c2dc0e97c0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:04.492576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f2da6-bd9c-4d9f-b352-b0e0ee0d170f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:04.494098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 581/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Aug 10 13:27:04 2022] GET /identity/v3/domains/0d9185d8fc844a909ba842c2dc0e97c0 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:04.499702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcc8c6-b863-4b89-a8ef-a9a5ecad6532 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:04.573383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:04.595686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcc8c6-b863-4b89-a8ef-a9a5ecad6532 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:04.628623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b9acf-bd25-4dc7-9ae5-be223b760a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:04.629209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b9acf-bd25-4dc7-9ae5-be223b760a49 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:04.629510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b9acf-bd25-4dc7-9ae5-be223b760a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:04.629776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b9acf-bd25-4dc7-9ae5-be223b760a49 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:04.630411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b9acf-bd25-4dc7-9ae5-be223b760a49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:04.631817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b9acf-bd25-4dc7-9ae5-be223b760a49 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:04.652151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 596/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:27:04 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:27:06.661462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dddc53-bf8a-4d6c-9ae4-162608bc672c None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:06.661997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dddc53-bf8a-4d6c-9ae4-162608bc672c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:06.661997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dddc53-bf8a-4d6c-9ae4-162608bc672c None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:06.663341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 582/1178] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:27:06.668345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b20d76-693a-42ad-8aa1-cbd575d8d237 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:06.668775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b20d76-693a-42ad-8aa1-cbd575d8d237 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:06.668873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b20d76-693a-42ad-8aa1-cbd575d8d237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:06.686907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-35b20d76-693a-42ad-8aa1-cbd575d8d237 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:06.688076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-35b20d76-693a-42ad-8aa1-cbd575d8d237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:06.752584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 597/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:27:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:27:06.775481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dddc53-bf8a-4d6c-9ae4-162608bc672c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:06.868918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:06.873970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dddc53-bf8a-4d6c-9ae4-162608bc672c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:06.906351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8e56-f2c3-4d18-8f18-ccbcea7f2f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:06.906961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8e56-f2c3-4d18-8f18-ccbcea7f2f95 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:06.907236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8e56-f2c3-4d18-8f18-ccbcea7f2f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:06.907503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8e56-f2c3-4d18-8f18-ccbcea7f2f95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:06.917645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a8e56-f2c3-4d18-8f18-ccbcea7f2f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-cd3a8e56-f2c3-4d18-8f18-ccbcea7f2f95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:27:06.922157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:27:06.928700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:27:06.929874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 583/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:27:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:27:06.930795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b20d76-693a-42ad-8aa1-cbd575d8d237 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:06.989027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:07.027323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b20d76-693a-42ad-8aa1-cbd575d8d237 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:07.059705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:07.060291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:07.060568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:07.060834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:07.061515 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:27:07.061811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:07.063162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:07.070838 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 598/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:27:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:07.076457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8e56-f2c3-4d18-8f18-ccbcea7f2f95 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:07.174191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8e56-f2c3-4d18-8f18-ccbcea7f2f95 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:07.206202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc43aa-7d45-4edd-9a4b-70a160c2d8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:07.206812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc43aa-7d45-4edd-9a4b-70a160c2d8ff None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:07.207086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc43aa-7d45-4edd-9a4b-70a160c2d8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:07.207355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc43aa-7d45-4edd-9a4b-70a160c2d8ff None admin] PATH_INFO: `/v3/users/d9a4ebc63f5b4f8a9893349f0273041e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:07.219694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc43aa-7d45-4edd-9a4b-70a160c2d8ff None admin] RBAC: Authorizing `identity:get_user(user_id=d9a4ebc63f5b4f8a9893349f0273041e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:07.221733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc43aa-7d45-4edd-9a4b-70a160c2d8ff None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:07.223325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 584/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:27:07 2022] GET /identity/v3/users/d9a4ebc63f5b4f8a9893349f0273041e => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:07.228983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:07.323906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214890-cec6-4b0b-a901-9f955e2bead8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:07.346033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:07.356554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8589d91-e4da-45e9-90fc-d1701b97622f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:07.357145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8589d91-e4da-45e9-90fc-d1701b97622f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:07.357446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8589d91-e4da-45e9-90fc-d1701b97622f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:07.357968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8589d91-e4da-45e9-90fc-d1701b97622f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:07.425399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8589d91-e4da-45e9-90fc-d1701b97622f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:07.426744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8589d91-e4da-45e9-90fc-d1701b97622f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:07.522030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 599/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:27:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:27:07.527888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc43aa-7d45-4edd-9a4b-70a160c2d8ff None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:07.545532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:07.625179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc43aa-7d45-4edd-9a4b-70a160c2d8ff None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:07.657903 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039a8d4-9698-45f2-bc07-ad956c5b68c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:07.658515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039a8d4-9698-45f2-bc07-ad956c5b68c8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:07.659331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039a8d4-9698-45f2-bc07-ad956c5b68c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:07.659594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039a8d4-9698-45f2-bc07-ad956c5b68c8 None admin] PATH_INFO: `/v3/projects/66d165d3a5a34a189a05132c700ebb9f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:07.660687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039a8d4-9698-45f2-bc07-ad956c5b68c8 None admin] RBAC: Authorizing `identity:get_project(project_id=66d165d3a5a34a189a05132c700ebb9f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:07.662116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039a8d4-9698-45f2-bc07-ad956c5b68c8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:07.663700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 585/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:27:07 2022] GET /identity/v3/projects/66d165d3a5a34a189a05132c700ebb9f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:27:07.669750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8589d91-e4da-45e9-90fc-d1701b97622f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:07.694016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:07.765092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8589d91-e4da-45e9-90fc-d1701b97622f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:07.797824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3e27f-7dd2-41bb-a7c8-1ca4f6a1f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:07.798406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e27f-7dd2-41bb-a7c8-1ca4f6a1f275 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:07.798677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e27f-7dd2-41bb-a7c8-1ca4f6a1f275 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:07.798939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3e27f-7dd2-41bb-a7c8-1ca4f6a1f275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:07.800187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3e27f-7dd2-41bb-a7c8-1ca4f6a1f275 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=66d165d3a5a34a189a05132c700ebb9f, user.id=d9a4ebc63f5b4f8a9893349f0273041e) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:27:07.800477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3e27f-7dd2-41bb-a7c8-1ca4f6a1f275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:07.801900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3e27f-7dd2-41bb-a7c8-1ca4f6a1f275 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:07.806957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 600/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:27:07 2022] GET /identity/v3/role_assignments?user.id=d9a4ebc63f5b4f8a9893349f0273041e&scope.project.id=66d165d3a5a34a189a05132c700ebb9f&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:09.757827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73686783-6611-4ce4-8e73-937140f33afa None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:09.758379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73686783-6611-4ce4-8e73-937140f33afa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:09.758379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73686783-6611-4ce4-8e73-937140f33afa None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:09.759678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 586/1186] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:27:09.764718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36df0f-9760-48ef-915c-de4cff3ca558 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:09.765131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36df0f-9760-48ef-915c-de4cff3ca558 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:09.765295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36df0f-9760-48ef-915c-de4cff3ca558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:09.784053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8f36df0f-9760-48ef-915c-de4cff3ca558 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:09.785261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8f36df0f-9760-48ef-915c-de4cff3ca558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:09.849034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 601/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:27:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:27:09.870890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73686783-6611-4ce4-8e73-937140f33afa None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:09.951010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:09.967471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73686783-6611-4ce4-8e73-937140f33afa None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:09.999497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dcde4-1d7a-4fdc-b8ff-13057a313c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:10.000102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dcde4-1d7a-4fdc-b8ff-13057a313c73 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:10.000373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dcde4-1d7a-4fdc-b8ff-13057a313c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:10.000635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dcde4-1d7a-4fdc-b8ff-13057a313c73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:10.010691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dcde4-1d7a-4fdc-b8ff-13057a313c73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-122dcde4-1d7a-4fdc-b8ff-13057a313c73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:27:10.015154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:27:10.020861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:27:10.021936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 587/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:27:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:27:10.022797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36df0f-9760-48ef-915c-de4cff3ca558 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:10.039562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:10.117130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36df0f-9760-48ef-915c-de4cff3ca558 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:10.150080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:10.150665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:10.150939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:10.151206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:10.151859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:27:10.152154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:10.153538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:10.161194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 602/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:27:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:10.166887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dcde4-1d7a-4fdc-b8ff-13057a313c73 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:10.262834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dcde4-1d7a-4fdc-b8ff-13057a313c73 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:10.295068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34492e-4130-4194-b7d4-6836475efec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:10.295684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34492e-4130-4194-b7d4-6836475efec6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:10.295955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34492e-4130-4194-b7d4-6836475efec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:10.296215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34492e-4130-4194-b7d4-6836475efec6 None admin] PATH_INFO: `/v3/users/d9a4ebc63f5b4f8a9893349f0273041e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:10.297327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34492e-4130-4194-b7d4-6836475efec6 None admin] RBAC: Authorizing `identity:get_user(user_id=d9a4ebc63f5b4f8a9893349f0273041e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:10.298725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34492e-4130-4194-b7d4-6836475efec6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:10.300874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 588/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:27:10 2022] GET /identity/v3/users/d9a4ebc63f5b4f8a9893349f0273041e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:10.306668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:10.383986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:10.402459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036507fd-8027-4423-a8ac-3e18e6f02d18 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:10.435004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9811a5d-554a-4c4c-8ec6-2a815439a6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:10.435606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811a5d-554a-4c4c-8ec6-2a815439a6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:10.435877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811a5d-554a-4c4c-8ec6-2a815439a6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:10.436141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811a5d-554a-4c4c-8ec6-2a815439a6a3 None admin] PATH_INFO: `/v3/projects/66d165d3a5a34a189a05132c700ebb9f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:10.437223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9811a5d-554a-4c4c-8ec6-2a815439a6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=66d165d3a5a34a189a05132c700ebb9f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:10.438655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9811a5d-554a-4c4c-8ec6-2a815439a6a3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:10.440255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 603/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:27:10 2022] GET /identity/v3/projects/66d165d3a5a34a189a05132c700ebb9f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:10.446167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34492e-4130-4194-b7d4-6836475efec6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:10.460875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:10.544369 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34492e-4130-4194-b7d4-6836475efec6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:10.576160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507162a-7b7a-428f-b49c-81b225d7206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:10.576801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507162a-7b7a-428f-b49c-81b225d7206e None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:10.577075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507162a-7b7a-428f-b49c-81b225d7206e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:10.577370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507162a-7b7a-428f-b49c-81b225d7206e None admin] PATH_INFO: `/v3/projects/66d165d3a5a34a189a05132c700ebb9f/users/d9a4ebc63f5b4f8a9893349f0273041e/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:10.579224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507162a-7b7a-428f-b49c-81b225d7206e None admin] RBAC: Authorizing `identity:create_grant(project_id=66d165d3a5a34a189a05132c700ebb9f, user_id=d9a4ebc63f5b4f8a9893349f0273041e, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:10.580670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507162a-7b7a-428f-b49c-81b225d7206e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:10.592606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 589/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:27:10 2022] PUT /identity/v3/projects/66d165d3a5a34a189a05132c700ebb9f/users/d9a4ebc63f5b4f8a9893349f0273041e/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:27:12.543857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcffa73-bab4-47e0-8262-b3ec88122c7b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:12.544357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcffa73-bab4-47e0-8262-b3ec88122c7b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:12.544357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcffa73-bab4-47e0-8262-b3ec88122c7b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:12.545707 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 604/1193] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:27:12.550667 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4902a4-44cb-439f-9035-7c168434ebaa None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:12.551110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4902a4-44cb-439f-9035-7c168434ebaa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:12.551202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4902a4-44cb-439f-9035-7c168434ebaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:12.569196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ac4902a4-44cb-439f-9035-7c168434ebaa None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:27:12.570399 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ac4902a4-44cb-439f-9035-7c168434ebaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:12.633144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 590/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:27:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:27:12.656237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcffa73-bab4-47e0-8262-b3ec88122c7b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:12.753128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcffa73-bab4-47e0-8262-b3ec88122c7b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:12.784798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf97bb-bb3f-488e-ae02-44dc0ad14730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:12.785616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf97bb-bb3f-488e-ae02-44dc0ad14730 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:12.785890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf97bb-bb3f-488e-ae02-44dc0ad14730 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:12.786154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf97bb-bb3f-488e-ae02-44dc0ad14730 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:12.797531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf97bb-bb3f-488e-ae02-44dc0ad14730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d6bf97bb-bb3f-488e-ae02-44dc0ad14730 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:27:12.802869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:27:12.803988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:27:12.804963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 605/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:27:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:27:12.810502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4902a4-44cb-439f-9035-7c168434ebaa None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:12.812683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:12.917723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4902a4-44cb-439f-9035-7c168434ebaa None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:12.950649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:12.951251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:12.951529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:12.951798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:12.952450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:27:12.952751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:12.954128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:12.962774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 591/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:12.968674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf97bb-bb3f-488e-ae02-44dc0ad14730 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:12.979582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:13.064699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf97bb-bb3f-488e-ae02-44dc0ad14730 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:13.097735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab27cc3-15c1-43c5-9b48-e9a5c80bba50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:13.098297 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab27cc3-15c1-43c5-9b48-e9a5c80bba50 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:13.098574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab27cc3-15c1-43c5-9b48-e9a5c80bba50 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:13.098844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab27cc3-15c1-43c5-9b48-e9a5c80bba50 None admin] PATH_INFO: `/v3/users/d9a4ebc63f5b4f8a9893349f0273041e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:13.099938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab27cc3-15c1-43c5-9b48-e9a5c80bba50 None admin] RBAC: Authorizing `identity:get_user(user_id=d9a4ebc63f5b4f8a9893349f0273041e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:13.101389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab27cc3-15c1-43c5-9b48-e9a5c80bba50 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:13.102937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 606/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 10 13:27:12 2022] GET /identity/v3/users/d9a4ebc63f5b4f8a9893349f0273041e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:13.108899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:13.174241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:13.211294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1045-6519-4618-b3b5-7dd944496607 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:13.243542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264aed4-d72d-4b2e-9c6f-4e602cf12e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:13.244142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264aed4-d72d-4b2e-9c6f-4e602cf12e9e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:13.244416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264aed4-d72d-4b2e-9c6f-4e602cf12e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:13.244686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264aed4-d72d-4b2e-9c6f-4e602cf12e9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:13.309136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264aed4-d72d-4b2e-9c6f-4e602cf12e9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:13.310598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264aed4-d72d-4b2e-9c6f-4e602cf12e9e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:13.368373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:13.406616 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 592/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:27:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 13:27:13.413067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab27cc3-15c1-43c5-9b48-e9a5c80bba50 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:13.487888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:13.509642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab27cc3-15c1-43c5-9b48-e9a5c80bba50 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:13.541377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1d333-afb8-4558-9c42-562c79fab4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:13.541987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1d333-afb8-4558-9c42-562c79fab4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:13.542263 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1d333-afb8-4558-9c42-562c79fab4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:13.542531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1d333-afb8-4558-9c42-562c79fab4d9 None admin] PATH_INFO: `/v3/projects/66d165d3a5a34a189a05132c700ebb9f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:13.543623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1d333-afb8-4558-9c42-562c79fab4d9 None admin] RBAC: Authorizing `identity:get_project(project_id=66d165d3a5a34a189a05132c700ebb9f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:13.545036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1d333-afb8-4558-9c42-562c79fab4d9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:13.546668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 607/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 10 13:27:13 2022] GET /identity/v3/projects/66d165d3a5a34a189a05132c700ebb9f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:27:13.553296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264aed4-d72d-4b2e-9c6f-4e602cf12e9e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:13.647268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264aed4-d72d-4b2e-9c6f-4e602cf12e9e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:13.679839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf553507-65be-41d2-b75d-be4634088270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:13.680421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553507-65be-41d2-b75d-be4634088270 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:13.680692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553507-65be-41d2-b75d-be4634088270 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:13.680958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553507-65be-41d2-b75d-be4634088270 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:13.681687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf553507-65be-41d2-b75d-be4634088270 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=66d165d3a5a34a189a05132c700ebb9f, user.id=d9a4ebc63f5b4f8a9893349f0273041e) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:27:13.681986 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf553507-65be-41d2-b75d-be4634088270 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:13.683381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf553507-65be-41d2-b75d-be4634088270 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:13.688891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 593/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:27:13 2022] GET /identity/v3/role_assignments?user.id=d9a4ebc63f5b4f8a9893349f0273041e&scope.project.id=66d165d3a5a34a189a05132c700ebb9f&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:27:23.537547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=28 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:23.686331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=86 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:08.973746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989737-d963-476e-b7ee-6fb6023e4ace None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:08.973746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989737-d963-476e-b7ee-6fb6023e4ace None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:08.973746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989737-d963-476e-b7ee-6fb6023e4ace None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:08.975523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 608/1201] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:08.983084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23bd6-92f0-4ce4-825a-d064f3fa3e7f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:08.983786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23bd6-92f0-4ce4-825a-d064f3fa3e7f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:08.983786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23bd6-92f0-4ce4-825a-d064f3fa3e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:09.014605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c4e23bd6-92f0-4ce4-825a-d064f3fa3e7f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:09.015890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c4e23bd6-92f0-4ce4-825a-d064f3fa3e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:09.112486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 594/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:09.140134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04989737-d963-476e-b7ee-6fb6023e4ace None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:09.286097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04989737-d963-476e-b7ee-6fb6023e4ace None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:09.328458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:09.329201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:09.329562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:09.330041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:09.344604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:09.346446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:09.364987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:09.373693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:09.374943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:09.374943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:09.374943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:09.374943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 609/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:09.378854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e23bd6-92f0-4ce4-825a-d064f3fa3e7f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:09.512171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e23bd6-92f0-4ce4-825a-d064f3fa3e7f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:09.555026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:09.561698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef38e7-0ab3-490d-919e-9a8379e770b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:09.562342 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef38e7-0ab3-490d-919e-9a8379e770b6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:09.562651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef38e7-0ab3-490d-919e-9a8379e770b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:09.562923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef38e7-0ab3-490d-919e-9a8379e770b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:09.563531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef38e7-0ab3-490d-919e-9a8379e770b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:09.563891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef38e7-0ab3-490d-919e-9a8379e770b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:09.565315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef38e7-0ab3-490d-919e-9a8379e770b6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:09.578440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 595/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:09.584213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:09.645772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:09.706125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75b504-b83c-43da-84e9-ce52f3052a3d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:09.749275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bcab4-0c5a-42a4-8dab-03b94b762f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:09.749877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bcab4-0c5a-42a4-8dab-03b94b762f6f None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:09.750192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bcab4-0c5a-42a4-8dab-03b94b762f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:09.750432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bcab4-0c5a-42a4-8dab-03b94b762f6f None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:09.751076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bcab4-0c5a-42a4-8dab-03b94b762f6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:09.753123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bcab4-0c5a-42a4-8dab-03b94b762f6f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:09.788551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 610/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 10 13:28:09 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:28:11.976258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfa8e4-5a8b-4a25-812c-ea6638f699b0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:11.976811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfa8e4-5a8b-4a25-812c-ea6638f699b0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:11.976811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfa8e4-5a8b-4a25-812c-ea6638f699b0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:11.978292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 596/1206] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:11.983923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22418a0-16ca-447b-b59b-8b53b82f2592 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:11.984449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22418a0-16ca-447b-b59b-8b53b82f2592 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:11.984449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22418a0-16ca-447b-b59b-8b53b82f2592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:12.003162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c22418a0-16ca-447b-b59b-8b53b82f2592 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:12.004349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c22418a0-16ca-447b-b59b-8b53b82f2592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:12.076114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 611/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 13:28:12.099959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfa8e4-5a8b-4a25-812c-ea6638f699b0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:12.213771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfa8e4-5a8b-4a25-812c-ea6638f699b0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:12.259951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea640f-3e16-4a90-8c69-cef88b942839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:12.260512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea640f-3e16-4a90-8c69-cef88b942839 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:12.260786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea640f-3e16-4a90-8c69-cef88b942839 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:12.261050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea640f-3e16-4a90-8c69-cef88b942839 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:12.272707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea640f-3e16-4a90-8c69-cef88b942839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-4cea640f-3e16-4a90-8c69-cef88b942839 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:28:12.277409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:12.278396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:28:12.279280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:12.283692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 597/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:28:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:12.284871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22418a0-16ca-447b-b59b-8b53b82f2592 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:12.328565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:12.395933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22418a0-16ca-447b-b59b-8b53b82f2592 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:12.447902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:12.448721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:12.449098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:12.449505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:12.453279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:12.453279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:12.453279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:12.461894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 612/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:28:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:12.467593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea640f-3e16-4a90-8c69-cef88b942839 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:12.527147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:12.583934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea640f-3e16-4a90-8c69-cef88b942839 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:12.621329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:12.621953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:12.622221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:12.622491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:12.633112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:12.634726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:12.651925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:12.653013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:12.653965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:12.653965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:12.653965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:12.653965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 598/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:28:12.659502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:12.793533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:12.797514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6003475-be93-4a6e-a087-f7bdb93cfd34 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:12.844167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:12.845864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:12.845864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:12.846153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:12.846716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:12.847008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:12.848407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:12.863872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 613/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:12.869387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:12.978173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b756913-b2b9-4695-a0b7-6c636247f8d5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:12.979809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:13.013836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:13.014867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:13.015358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:13.015966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] PATH_INFO: `/v3/users/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:13.024102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:13.025579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:28:13.038183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:13.039340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:13.043190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:13.045704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 599/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:13.045834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:13.160058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b23f4-c829-408d-b06b-8e99735dbc06 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:13.197089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:13.197759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:13.198034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:13.198302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:13.199025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:13.199320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:13.200735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:13.220428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 614/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 10 13:28:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:13.226125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:13.353132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6d265-099e-49d2-a952-84fa9255dcac None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:13.391148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477989d-d935-41dd-a75f-c790f5c88319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:13.391779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477989d-d935-41dd-a75f-c790f5c88319 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:13.392131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477989d-d935-41dd-a75f-c790f5c88319 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:13.392410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477989d-d935-41dd-a75f-c790f5c88319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:13.425922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:13.475419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477989d-d935-41dd-a75f-c790f5c88319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:13.477318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477989d-d935-41dd-a75f-c790f5c88319 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:13.600405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 600/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:28:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:28:13.606280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:13.630624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:13.718496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95023a42-70a7-4aef-88ec-177c5a1d8c87 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:13.760543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:13.761171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:13.761473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:13.761741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:13.777337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:13.779203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:13.797217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:13.799803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:13.800733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:13.800733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:13.800733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:13.803789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 615/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:28:13.805503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477989d-d935-41dd-a75f-c790f5c88319 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:13.873864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:13.915027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477989d-d935-41dd-a75f-c790f5c88319 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:13.955557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:13.956178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:13.956469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:13.956725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:13.957395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:13.957695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:13.959051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:13.971373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 601/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:13.976785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:14.058833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:14.092206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef38ca-8ff8-4823-aece-e2b51f60940f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:14.129853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:14.130482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:14.131018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:14.131141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:14.138728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:14.140186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:28:14.152915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:14.154127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:14.155444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:28:14.155444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:28:14.155444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:28:14.155444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:28:14.155444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:28:14.155444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:28:14.155444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:14.155444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:14.159705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 616/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:28:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:14.160557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:14.281770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4125-501c-4230-b7c3-ff42c4a5b169 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:14.314364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:14.327456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8e4e4-9193-4b76-a308-e43e30135b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:14.328160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8e4e4-9193-4b76-a308-e43e30135b46 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:14.328507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8e4e4-9193-4b76-a308-e43e30135b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:14.328792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8e4e4-9193-4b76-a308-e43e30135b46 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:14.329540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8e4e4-9193-4b76-a308-e43e30135b46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:14.329837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8e4e4-9193-4b76-a308-e43e30135b46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:14.331279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8e4e4-9193-4b76-a308-e43e30135b46 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:14.345421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 602/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:28:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:28:14.351299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:14.462654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142e323-dee1-4476-b3ec-d5f88db13999 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:14.492309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:14.499451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236214a4-8fc6-4534-9985-7e4871e32ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:14.500118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236214a4-8fc6-4534-9985-7e4871e32ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:14.500462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236214a4-8fc6-4534-9985-7e4871e32ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:14.500796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236214a4-8fc6-4534-9985-7e4871e32ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:14.501669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236214a4-8fc6-4534-9985-7e4871e32ab0 None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=e6a31a4ad1f947308df7e4de047e85ce) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:14.502030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236214a4-8fc6-4534-9985-7e4871e32ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:14.503517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236214a4-8fc6-4534-9985-7e4871e32ab0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:14.509183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 617/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:28:14 2022] GET /identity/v3/role_assignments?user.id=e6a31a4ad1f947308df7e4de047e85ce&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:16.942238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1a99-84bd-4ba7-9cfc-5732ab8c722e None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:16.943057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1a99-84bd-4ba7-9cfc-5732ab8c722e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:16.943057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1a99-84bd-4ba7-9cfc-5732ab8c722e None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:16.944108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 603/1220] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:16.949917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21cd29-45f9-4c5b-9c30-91cedc7990ff None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:16.950365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21cd29-45f9-4c5b-9c30-91cedc7990ff None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:16.950815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21cd29-45f9-4c5b-9c30-91cedc7990ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:16.972628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7b21cd29-45f9-4c5b-9c30-91cedc7990ff None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:16.974220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7b21cd29-45f9-4c5b-9c30-91cedc7990ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:17.052627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 618/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:17.076612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1a99-84bd-4ba7-9cfc-5732ab8c722e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:17.185428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1a99-84bd-4ba7-9cfc-5732ab8c722e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:17.222873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d3eee-9c28-4eb3-93f3-e034fa1f01d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:17.223451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d3eee-9c28-4eb3-93f3-e034fa1f01d9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:17.223733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d3eee-9c28-4eb3-93f3-e034fa1f01d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:17.223994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d3eee-9c28-4eb3-93f3-e034fa1f01d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:17.234710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d3eee-9c28-4eb3-93f3-e034fa1f01d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ff3d3eee-9c28-4eb3-93f3-e034fa1f01d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:28:17.239918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:17.246251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:17.247371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 604/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:28:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:17.248321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21cd29-45f9-4c5b-9c30-91cedc7990ff None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:17.352981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21cd29-45f9-4c5b-9c30-91cedc7990ff None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:17.377619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:17.389772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:17.390455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:17.390750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:17.391059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:17.391812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:17.392647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:17.394175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:17.403058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 619/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:28:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:17.410759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d3eee-9c28-4eb3-93f3-e034fa1f01d9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:17.461135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:17.523592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d3eee-9c28-4eb3-93f3-e034fa1f01d9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:17.561711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:17.562343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:17.562619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:17.562886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:17.572709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:17.574194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:17.588943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:17.590159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:17.591088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:17.591088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:17.591088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:17.591088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 605/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:17.596567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:17.707617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa60de-a55a-4468-b8ff-3959850f0d33 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:17.745939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:17.746549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:17.746826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:17.747115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:17.747753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:17.748047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:17.749477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:17.761346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 620/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:17.768443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:17.882021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:17.889441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd35df-d387-438c-be52-675ad422001d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:17.924513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:17.925212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:17.925692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:17.926084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:17.935394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:17.937415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:28:17.952277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:17.953926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:17.955713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:17.959759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 606/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:17.960980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:18.008112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:18.061533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37b4b0-16ef-40c4-b217-b10c2c277e42 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:18.097403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:18.098006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:18.098275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:18.098572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:18.099344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:18.099648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:18.101150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:18.119102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 621/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 10 13:28:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:18.124861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:18.263291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bde4b-e525-4ea8-ad16-93fff2b5e287 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:18.300464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:18.301096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:18.301396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:18.301662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:18.313536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:18.314985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:18.330365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:18.331588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:18.332710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:18.332710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:18.332710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:18.332710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 607/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:28:18.337917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:18.459569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e41d74-b668-41c5-8cfa-9d21dbfa01ff None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:18.462182 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:18.497113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:18.497807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:18.498071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:18.498286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:18.498933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:18.499227 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:18.500592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:18.513907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 622/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:18.520882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:18.531013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:18.631001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c029b-48ba-4e79-b315-e2e5753430bd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:18.666697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:18.667427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:18.667697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:18.667993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:18.675657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:18.677247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:28:18.691604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:18.692644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:18.693550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 608/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:28:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:18.698678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:18.805624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885ef6d-07b4-4698-9b03-23b518074067 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:18.844935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ad52-7415-430c-b40f-d35e98272ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:18.845609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ad52-7415-430c-b40f-d35e98272ddd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:18.845919 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ad52-7415-430c-b40f-d35e98272ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:18.846260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ad52-7415-430c-b40f-d35e98272ddd None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:18.847055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3ad52-7415-430c-b40f-d35e98272ddd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:18.847386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3ad52-7415-430c-b40f-d35e98272ddd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:18.848948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3ad52-7415-430c-b40f-d35e98272ddd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:18.863216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 623/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:28:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:28:18.869911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:18.942858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:18.978186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce8c2d-7d5a-4cca-8f08-cd20b40de640 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:19.014044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fe842-f1f0-42d8-94dc-ac281c507abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:19.014699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fe842-f1f0-42d8-94dc-ac281c507abc None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:19.014975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fe842-f1f0-42d8-94dc-ac281c507abc None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:19.015242 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fe842-f1f0-42d8-94dc-ac281c507abc None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/e6a31a4ad1f947308df7e4de047e85ce/roles/a9f22cef53e740cdbc09d1137362031e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:19.031847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93fe842-f1f0-42d8-94dc-ac281c507abc None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=e6a31a4ad1f947308df7e4de047e85ce, role_id=a9f22cef53e740cdbc09d1137362031e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:19.033505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93fe842-f1f0-42d8-94dc-ac281c507abc None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:19.045974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 609/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:28:18 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/e6a31a4ad1f947308df7e4de047e85ce/roles/a9f22cef53e740cdbc09d1137362031e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:28:21.145902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eec69-fcac-40c8-a5a4-6a8342855002 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:21.146448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eec69-fcac-40c8-a5a4-6a8342855002 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:21.146448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eec69-fcac-40c8-a5a4-6a8342855002 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:21.147906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 624/1233] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:21.153074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730da7c-8476-4cf6-b95f-a2d1f9e7c0c6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:21.153518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730da7c-8476-4cf6-b95f-a2d1f9e7c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:21.153864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730da7c-8476-4cf6-b95f-a2d1f9e7c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:21.174424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a730da7c-8476-4cf6-b95f-a2d1f9e7c0c6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:21.175648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a730da7c-8476-4cf6-b95f-a2d1f9e7c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:21.246659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 610/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:21.271309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890eec69-fcac-40c8-a5a4-6a8342855002 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:21.297085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:21.378179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890eec69-fcac-40c8-a5a4-6a8342855002 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:21.411596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7f93a-4965-4674-b658-9e69d0474e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:21.412143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f93a-4965-4674-b658-9e69d0474e08 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:21.412412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f93a-4965-4674-b658-9e69d0474e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:21.412674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f93a-4965-4674-b658-9e69d0474e08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:21.422514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7f93a-4965-4674-b658-9e69d0474e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-cbb7f93a-4965-4674-b658-9e69d0474e08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:28:21.427564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:21.428657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:21.429612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 625/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 10 13:28:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:21.434779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730da7c-8476-4cf6-b95f-a2d1f9e7c0c6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:21.482336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:21.542717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730da7c-8476-4cf6-b95f-a2d1f9e7c0c6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:21.578174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:21.578787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:21.579074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:21.579330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:21.579992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:21.580289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:21.581690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:21.589975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 611/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 10 13:28:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:21.596467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7f93a-4965-4674-b658-9e69d0474e08 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:21.685899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:21.702675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7f93a-4965-4674-b658-9e69d0474e08 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:21.738505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:21.739146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:21.739382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:21.739644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:21.750209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:21.752278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:21.766611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:21.768903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:21.770141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:21.770141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:21.770141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:21.772998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 626/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:28:21.774000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:21.899605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78b59a-0faa-4561-8680-733d4dc9de7b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:21.910890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:21.940624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:21.941157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:21.941523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:21.941735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:21.942493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:21.942805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:21.944241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:21.957469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 612/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:21.963335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:22.070116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cd2a8-7278-4f2f-8ccf-7acb4dc4da10 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:22.107316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:22.108193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:22.108470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:22.108737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:22.111470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:22.117404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:22.119439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:28:22.131541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:22.132788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:22.134596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 627/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:22.138542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:22.252929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31627b09-23da-4b9a-9522-ed1edcc7da9a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:22.291157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:22.291770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:22.292044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:22.292319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:22.293038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:22.293376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:22.294789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:22.317479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 613/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 10 13:28:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:22.318781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:22.422379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32db3bd-c53c-4f92-a76d-a33c08596c9f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:22.460228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afc1ae-7a21-4b39-8f7d-dc8a91267b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:22.460809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afc1ae-7a21-4b39-8f7d-dc8a91267b2b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:22.461086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afc1ae-7a21-4b39-8f7d-dc8a91267b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:22.461380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afc1ae-7a21-4b39-8f7d-dc8a91267b2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:22.522499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:22.529190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afc1ae-7a21-4b39-8f7d-dc8a91267b2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:22.530754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afc1ae-7a21-4b39-8f7d-dc8a91267b2b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:22.634236 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 628/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:28:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:28:22.640594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:22.673866 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:22.750496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc8883-7b50-47f6-a244-90fd31c08d72 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:22.792798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:22.793366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:22.793646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:22.794071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:22.805082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:22.806567 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:22.821906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:22.823071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:22.823990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:22.823990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:22.823990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:22.823990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 614/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:22.829614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afc1ae-7a21-4b39-8f7d-dc8a91267b2b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:22.931782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:22.938008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afc1ae-7a21-4b39-8f7d-dc8a91267b2b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:22.975614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:22.976210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:22.976516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:22.976752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:22.977423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:22.977718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:22.979083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:22.990858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 629/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:28:22.996639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:23.084170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:23.106804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d4dd0-bfb7-419d-ba08-01da0c3a3637 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:23.143413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:23.144074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:23.144335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:23.144601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:23.152162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:23.153683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:28:23.165427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:23.166518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:23.167425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 615/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:28:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:23.172353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:23.281471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f1041-5455-428b-bd9c-e627a1c90de3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:23.316627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97a597-927c-4eab-8ce4-334139dfba8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:23.317256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97a597-927c-4eab-8ce4-334139dfba8c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:23.317547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97a597-927c-4eab-8ce4-334139dfba8c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:23.317829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97a597-927c-4eab-8ce4-334139dfba8c None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:23.318502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97a597-927c-4eab-8ce4-334139dfba8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:23.318801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97a597-927c-4eab-8ce4-334139dfba8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:23.320218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97a597-927c-4eab-8ce4-334139dfba8c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:23.332443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 630/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:28:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:28:23.338635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:23.450377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25124538-33cb-4907-8f40-031966da3d68 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:23.490694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ea0fb-e32a-4f71-8a0e-56318b51788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:23.491378 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea0fb-e32a-4f71-8a0e-56318b51788a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:23.491655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea0fb-e32a-4f71-8a0e-56318b51788a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:23.491923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea0fb-e32a-4f71-8a0e-56318b51788a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:23.492643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea0fb-e32a-4f71-8a0e-56318b51788a None admin] RBAC: Adding query filter params (role.id=a9f22cef53e740cdbc09d1137362031e, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=e6a31a4ad1f947308df7e4de047e85ce) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:23.492947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea0fb-e32a-4f71-8a0e-56318b51788a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:23.494394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea0fb-e32a-4f71-8a0e-56318b51788a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:23.500210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 616/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:28:23 2022] GET /identity/v3/role_assignments?user.id=e6a31a4ad1f947308df7e4de047e85ce&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=a9f22cef53e740cdbc09d1137362031e => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:25.568812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc80ee-3db2-4f94-9a3d-6cc73a997c13 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:25.569340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc80ee-3db2-4f94-9a3d-6cc73a997c13 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:25.569340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc80ee-3db2-4f94-9a3d-6cc73a997c13 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:25.570681 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 631/1247] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:25.576018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6216884-2f75-4c87-9892-0a279b35a7d2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:25.576443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6216884-2f75-4c87-9892-0a279b35a7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:25.576537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6216884-2f75-4c87-9892-0a279b35a7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:25.583785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:25.597096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b6216884-2f75-4c87-9892-0a279b35a7d2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:25.598338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b6216884-2f75-4c87-9892-0a279b35a7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:25.670725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 617/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:25.695117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc80ee-3db2-4f94-9a3d-6cc73a997c13 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:25.713944 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:25.801072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc80ee-3db2-4f94-9a3d-6cc73a997c13 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:25.842377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856a57e-5a76-4246-a675-1e3def0dd6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:25.842934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856a57e-5a76-4246-a675-1e3def0dd6fc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:25.843325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856a57e-5a76-4246-a675-1e3def0dd6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:25.843571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856a57e-5a76-4246-a675-1e3def0dd6fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:25.856393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856a57e-5a76-4246-a675-1e3def0dd6fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c856a57e-5a76-4246-a675-1e3def0dd6fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:28:25.863833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:25.864878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:25.865964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 632/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:28:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:25.872000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6216884-2f75-4c87-9892-0a279b35a7d2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:25.986950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6216884-2f75-4c87-9892-0a279b35a7d2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:26.020389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:26.026489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:26.027107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:26.027380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:26.027646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:26.028351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:26.028643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:26.031391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:26.040883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 618/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:28:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:26.046770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856a57e-5a76-4246-a675-1e3def0dd6fc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:26.133821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:26.161620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856a57e-5a76-4246-a675-1e3def0dd6fc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:26.199799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:26.200429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:26.200702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:26.200966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:26.212224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:26.213783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:26.230366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:26.231575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:26.232728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:26.232728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:26.232728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:26.232728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 633/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:26.238213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:26.344256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6e14-cef9-4e7d-8c74-94f505244a57 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:26.383213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:26.383993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:26.384284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:26.384631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:26.385392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:26.385720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:26.387071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:26.399506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 619/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:26.405467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:26.525691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829e170-f01a-40aa-a8f2-bbfa45d20ce5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:26.560017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:26.570982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:26.571704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:26.571980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:26.572294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:26.580901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:26.582381 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:28:26.594091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:26.595878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:26.597024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 634/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:26.601589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:26.663824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:26.711351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb28d02-8b8a-4d94-86c2-7b48d0cd1b59 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:26.747648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:26.748269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:26.748543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:26.748814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:26.749573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:26.749869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:26.751851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:26.770565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 620/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 10 13:28:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:26.776439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:26.908509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8888be-bfc1-40e6-a342-44504e848970 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:26.952466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338350f-d2f2-47b6-a3ab-918233e3f996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:26.953011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338350f-d2f2-47b6-a3ab-918233e3f996 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:26.953391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338350f-d2f2-47b6-a3ab-918233e3f996 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:26.953696 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338350f-d2f2-47b6-a3ab-918233e3f996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:27.019586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:27.037483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338350f-d2f2-47b6-a3ab-918233e3f996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:27.038937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338350f-d2f2-47b6-a3ab-918233e3f996 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:27.145086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 635/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:28:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:28:27.151038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:27.268585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8b06-8dda-46dd-ac22-833b7ee7075b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:27.286749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:27.316760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:27.317421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:27.317714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:27.317957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:27.328017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:27.330075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:27.343904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:27.345658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:27.346560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:27.346560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:27.346560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:27.346560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 621/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:27.351245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338350f-d2f2-47b6-a3ab-918233e3f996 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:27.442284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:27.458740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338350f-d2f2-47b6-a3ab-918233e3f996 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:27.502121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:27.502786 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:27.503102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:27.503396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:27.504095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:27.504395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:27.505886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:27.523797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 636/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:27.525060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:27.636923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31393459-3bca-4bad-bcdf-4031979d5a08 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:27.674267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:27.674926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:27.675239 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:27.675546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:27.682563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:27.684022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:28:27.697340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:27.698439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:27.699403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 622/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:28:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:27.707322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:27.823924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085496b-2f19-4522-9192-4738eacb5c8c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:27.865257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c37818-0a0f-45bc-95bf-33b570ddd14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:27.866208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37818-0a0f-45bc-95bf-33b570ddd14b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:27.866518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37818-0a0f-45bc-95bf-33b570ddd14b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:27.866805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37818-0a0f-45bc-95bf-33b570ddd14b None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:27.868095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37818-0a0f-45bc-95bf-33b570ddd14b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:27.868501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37818-0a0f-45bc-95bf-33b570ddd14b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:27.870466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37818-0a0f-45bc-95bf-33b570ddd14b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:27.886761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 637/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:28:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:27.896362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:27.927851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:28.020272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbb776-6141-4e35-84dd-5734a120989f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:28.059058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14470a-1a20-4202-9a4e-503e401c711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:28.059661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14470a-1a20-4202-9a4e-503e401c711f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:28.059935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14470a-1a20-4202-9a4e-503e401c711f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:28.060203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14470a-1a20-4202-9a4e-503e401c711f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:28.060912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14470a-1a20-4202-9a4e-503e401c711f None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=e6a31a4ad1f947308df7e4de047e85ce) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:28.061215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14470a-1a20-4202-9a4e-503e401c711f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:28.062748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14470a-1a20-4202-9a4e-503e401c711f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:28.068960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 623/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:28:27 2022] GET /identity/v3/role_assignments?user.id=e6a31a4ad1f947308df7e4de047e85ce&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:30.117595 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811479c5-4df0-4cea-9677-2345db19c6ff None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:30.118129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811479c5-4df0-4cea-9677-2345db19c6ff None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:30.118129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811479c5-4df0-4cea-9677-2345db19c6ff None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:30.119480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 638/1261] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:30.124542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbeb5d-c626-4693-a83a-0bb814f5371b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:30.124990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbeb5d-c626-4693-a83a-0bb814f5371b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:30.125093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbeb5d-c626-4693-a83a-0bb814f5371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:30.143625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-14dbeb5d-c626-4693-a83a-0bb814f5371b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:30.144789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-14dbeb5d-c626-4693-a83a-0bb814f5371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:30.209392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 624/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:30.231341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811479c5-4df0-4cea-9677-2345db19c6ff None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:30.242565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:30.329899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811479c5-4df0-4cea-9677-2345db19c6ff None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:30.362173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63871ac3-765c-4798-bad8-4a55239383ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:30.362783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871ac3-765c-4798-bad8-4a55239383ad None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:30.363057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871ac3-765c-4798-bad8-4a55239383ad None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:30.363322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871ac3-765c-4798-bad8-4a55239383ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:30.372902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63871ac3-765c-4798-bad8-4a55239383ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-63871ac3-765c-4798-bad8-4a55239383ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:28:30.378024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:30.379108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:30.380118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 639/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:28:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:30.384989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbeb5d-c626-4693-a83a-0bb814f5371b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:30.402531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:30.479697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbeb5d-c626-4693-a83a-0bb814f5371b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:30.512823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:30.513446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:30.513724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:30.513992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:30.514641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:30.514941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:30.516315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:30.524002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 625/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:28:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:30.529645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63871ac3-765c-4798-bad8-4a55239383ad None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:30.593284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:30.625876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63871ac3-765c-4798-bad8-4a55239383ad None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:30.658290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:30.658893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:30.659166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:30.659432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:30.669007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:30.670464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:30.684674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:30.685796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:30.686908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:30.686908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:30.686908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:30.686908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 640/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:30.691373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:30.769427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:30.789348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fb19f-723f-4390-8c03-5a25b79b163b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:30.825356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:30.825998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:30.826362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:30.826657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:30.827382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:30.827740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:30.829276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:30.841836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 626/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:30.848509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:30.949027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2d652-c918-40a4-b080-0362eff5599f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:30.968299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:30.983565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:30.984180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:30.984457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:30.984727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:30.992093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:30.993552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:28:31.004442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:31.005565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:31.006903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 641/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 13:28:31.011315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:31.106379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030e4a4-c6c3-4743-9be0-117ae6456b80 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:31.138338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:31.138925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:31.139186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:31.139456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:31.140160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:31.140460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:31.142395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:31.150653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:31.159662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 627/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 10 13:28:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:31.165119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:31.262617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd9988-8ce4-4f65-b9bf-ec56c727b613 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:31.296615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:31.297253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:31.297559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:31.297823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:31.308583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:31.310034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:31.323990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:31.324943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:31.326064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:31.326064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:31.326064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:31.326064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 642/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:31.331398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:31.426744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148ace6-2619-44cb-9ab9-a2f4734a3a03 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:31.459692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:31.460289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:31.460565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:31.460832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:31.461490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:31.461786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:31.463141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:31.473845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 628/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:31.479272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:31.510102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:31.584885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bd50e-8d6b-43be-b111-2db8cabcc2e7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:31.618210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:31.618831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:31.619108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:31.619377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:31.626411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:31.627850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:28:31.638663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:31.639724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:31.640762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 643/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:28:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:31.646188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:31.702994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:31.744084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6dd0-6ad3-4228-8a08-46e45ea379cf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:31.776334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd6d5e-8245-4863-a17c-158cd85e224e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:31.776927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6d5e-8245-4863-a17c-158cd85e224e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:31.777196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6d5e-8245-4863-a17c-158cd85e224e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:31.777489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6d5e-8245-4863-a17c-158cd85e224e None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:31.778160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6d5e-8245-4863-a17c-158cd85e224e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:31.778454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6d5e-8245-4863-a17c-158cd85e224e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:31.779871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6d5e-8245-4863-a17c-158cd85e224e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:31.794105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 629/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:28:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:31.800653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:31.894168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:31.903104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac830f-f096-44a6-8d27-df43e51d3e8c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:31.935540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086b030-ed41-46a2-b002-465dfbf435b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:31.936183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086b030-ed41-46a2-b002-465dfbf435b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:31.936462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086b030-ed41-46a2-b002-465dfbf435b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:31.936733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086b030-ed41-46a2-b002-465dfbf435b4 None admin] PATH_INFO: `/v3/projects/c5c8005917984c0d8397df48b0822d92/users/e6a31a4ad1f947308df7e4de047e85ce/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:31.939168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086b030-ed41-46a2-b002-465dfbf435b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c8005917984c0d8397df48b0822d92, user_id=e6a31a4ad1f947308df7e4de047e85ce, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:31.940589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086b030-ed41-46a2-b002-465dfbf435b4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:31.954447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 644/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 10 13:28:31 2022] PUT /identity/v3/projects/c5c8005917984c0d8397df48b0822d92/users/e6a31a4ad1f947308df7e4de047e85ce/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:28:33.946654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645f7b8-a13e-4044-b268-b64ff5515146 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:33.947185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645f7b8-a13e-4044-b268-b64ff5515146 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:33.947185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645f7b8-a13e-4044-b268-b64ff5515146 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:33.948517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 630/1274] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:33.953534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd676147-72b3-4ac9-8c1e-d17a05748a90 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:33.954410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd676147-72b3-4ac9-8c1e-d17a05748a90 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:33.954570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd676147-72b3-4ac9-8c1e-d17a05748a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:33.972727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fd676147-72b3-4ac9-8c1e-d17a05748a90 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:33.973971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fd676147-72b3-4ac9-8c1e-d17a05748a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:34.044626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 645/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:34.068006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645f7b8-a13e-4044-b268-b64ff5515146 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:34.165578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645f7b8-a13e-4044-b268-b64ff5515146 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:34.182632 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:34.199136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18c856-ff1c-4803-8acb-b7eaa9092bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:34.199696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c856-ff1c-4803-8acb-b7eaa9092bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:34.199971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c856-ff1c-4803-8acb-b7eaa9092bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:34.200240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c856-ff1c-4803-8acb-b7eaa9092bd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:34.210471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18c856-ff1c-4803-8acb-b7eaa9092bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-8b18c856-ff1c-4803-8acb-b7eaa9092bd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 13:28:34.215069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:34.216182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:34.217390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 631/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 10 13:28:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:34.222689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd676147-72b3-4ac9-8c1e-d17a05748a90 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:34.299485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:34.319895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd676147-72b3-4ac9-8c1e-d17a05748a90 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:34.361930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:34.362523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:34.362794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:34.363061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:34.364056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:34.364496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:34.366729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:34.374469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 646/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 10 13:28:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:34.380705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18c856-ff1c-4803-8acb-b7eaa9092bd2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:34.478072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18c856-ff1c-4803-8acb-b7eaa9092bd2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:34.510324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:34.510933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:34.511206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:34.511474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:34.521393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:34.522828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:34.536560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:34.542008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:34.543147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:34.543147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:34.543147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:34.543147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 632/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:34.543428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:34.637980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb5fa3-b9fa-4162-9409-1ed2d2d044ec None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:34.669541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:34.670121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:34.670394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:34.670659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:34.671284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:34.671574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:34.673606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:34.685159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 647/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:34.690742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:34.720906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:34.787942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326db63-c217-4637-8f14-a30bc803de53 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:34.826939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:34.827563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:34.827837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:34.828107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:34.835853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:34.837303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 13:28:34.848480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:34.849629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:34.851033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 633/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:34.855443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:34.859407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:34.950218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1d2ac-1ce2-4d97-a11d-b98e0f1e4538 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:34.983394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:34.983978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:34.984228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:34.984491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:34.985320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:34.985614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:34.987012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:35.003073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 648/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 10 13:28:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:35.008713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:35.096106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:35.105672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf86d31-45f9-4feb-995d-74916896e3f0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:35.138103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9abad-064d-474b-8d38-d7ed84e7122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:35.138690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9abad-064d-474b-8d38-d7ed84e7122c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:35.138966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9abad-064d-474b-8d38-d7ed84e7122c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:35.139232 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9abad-064d-474b-8d38-d7ed84e7122c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:35.203161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9abad-064d-474b-8d38-d7ed84e7122c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:35.205135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9abad-064d-474b-8d38-d7ed84e7122c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:35.289198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:35.300808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 634/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 10 13:28:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 13:28:35.306530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:35.365619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:35.403300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bb680-8258-45a2-b87d-de15370fa191 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:35.435565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:35.436174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:35.436451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:35.436721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:35.447498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:35.448923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 13:28:35.463341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:35.464437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 13:28:35.465547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 13:28:35.465547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 13:28:35.465547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:35.465547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 649/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:35.470657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9abad-064d-474b-8d38-d7ed84e7122c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:35.565076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9abad-064d-474b-8d38-d7ed84e7122c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:35.598443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:35.599037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:35.599311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:35.599576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:35.600226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:35.600520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:35.601886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:35.616103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 635/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 10 13:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:35.621665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:35.717217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834cd2d-a09e-4551-baef-b073065ee0a0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:35.730671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:35.750230 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:35.750889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:35.751162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:35.751431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:35.757777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:35.759202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 13:28:35.770350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:35.771457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Aug 10 13:28:35.772649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 650/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 10 13:28:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:35.777132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:35.831042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:35.879193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d1bc-e834-4e37-870a-8d43865ef491 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:35.919573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a618b4f-650f-476a-b19c-a0e180f7a36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:35.920173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a618b4f-650f-476a-b19c-a0e180f7a36a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:35.920449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a618b4f-650f-476a-b19c-a0e180f7a36a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:35.920717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a618b4f-650f-476a-b19c-a0e180f7a36a None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:35.921426 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a618b4f-650f-476a-b19c-a0e180f7a36a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:35.921729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a618b4f-650f-476a-b19c-a0e180f7a36a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:35.923145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a618b4f-650f-476a-b19c-a0e180f7a36a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:35.934679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 636/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 10 13:28:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:35.941185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:36.040472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15a75-d58c-4ec7-86fe-58d9adabf67a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:36.072237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c735c-844b-468e-8335-a8679439e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:36.072812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c735c-844b-468e-8335-a8679439e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:36.073083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c735c-844b-468e-8335-a8679439e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:36.073379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c735c-844b-468e-8335-a8679439e5e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:36.074087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c735c-844b-468e-8335-a8679439e5e6 None admin] RBAC: Adding query filter params (role.id=f1cc6c5dc9634036b4adb0495566b562, scope.project.id=c5c8005917984c0d8397df48b0822d92, user.id=e6a31a4ad1f947308df7e4de047e85ce) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:36.074384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c735c-844b-468e-8335-a8679439e5e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:36.075795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c735c-844b-468e-8335-a8679439e5e6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:36.081969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 651/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 10 13:28:35 2022] GET /identity/v3/role_assignments?user.id=e6a31a4ad1f947308df7e4de047e85ce&scope.project.id=c5c8005917984c0d8397df48b0822d92&role.id=f1cc6c5dc9634036b4adb0495566b562 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:38.082250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6486-6889-4ca4-9bc5-264aef18a4f4 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:38.082814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6486-6889-4ca4-9bc5-264aef18a4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:38.082814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6486-6889-4ca4-9bc5-264aef18a4f4 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:38.084125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 637/1288] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:38.089291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca7c2d-1639-45e5-b914-658d8bd89670 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:38.089735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca7c2d-1639-45e5-b914-658d8bd89670 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:38.089840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca7c2d-1639-45e5-b914-658d8bd89670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:38.108888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7cca7c2d-1639-45e5-b914-658d8bd89670 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:38.110797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7cca7c2d-1639-45e5-b914-658d8bd89670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:38.138787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:38.176214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 652/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:38.197808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c6486-6889-4ca4-9bc5-264aef18a4f4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:38.294704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c6486-6889-4ca4-9bc5-264aef18a4f4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:38.305485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:38.327724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:38.328339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:38.328634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:38.328897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:38.329495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:38.330833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:38.339106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:38.340296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:38.341376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 638/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:28:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:38.346679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca7c2d-1639-45e5-b914-658d8bd89670 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:38.440934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca7c2d-1639-45e5-b914-658d8bd89670 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:38.473737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f817e53-a862-41a4-9148-7c10e981c828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:38.474333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f817e53-a862-41a4-9148-7c10e981c828 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:38.474612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f817e53-a862-41a4-9148-7c10e981c828 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:38.474879 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f817e53-a862-41a4-9148-7c10e981c828 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:38.475501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f817e53-a862-41a4-9148-7c10e981c828 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:38.475795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f817e53-a862-41a4-9148-7c10e981c828 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:38.477154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f817e53-a862-41a4-9148-7c10e981c828 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:38.481815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 653/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 10 13:28:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:38.487947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:38.582515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71007ac4-e226-4a4b-b5bb-7b202bf100c3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:38.614502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86db51-16e9-4bcf-8661-942c52c8973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:38.615027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86db51-16e9-4bcf-8661-942c52c8973a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:38.615297 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86db51-16e9-4bcf-8661-942c52c8973a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:38.615561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86db51-16e9-4bcf-8661-942c52c8973a None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:38.616164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86db51-16e9-4bcf-8661-942c52c8973a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:38.616453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86db51-16e9-4bcf-8661-942c52c8973a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:38.617832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86db51-16e9-4bcf-8661-942c52c8973a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:38.622025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 639/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 10 13:28:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:40.611424 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670caff-1220-429c-9961-fc9a962e44d9 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:40.611943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670caff-1220-429c-9961-fc9a962e44d9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:40.611943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670caff-1220-429c-9961-fc9a962e44d9 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:40.613251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 654/1293] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:40.618374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a64e7-14e9-4fcc-a945-b6df5986ca8a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:40.618757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a64e7-14e9-4fcc-a945-b6df5986ca8a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:40.618920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a64e7-14e9-4fcc-a945-b6df5986ca8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:40.636910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-259a64e7-14e9-4fcc-a945-b6df5986ca8a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:40.638137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-259a64e7-14e9-4fcc-a945-b6df5986ca8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:40.666852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:40.703958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 640/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:40.726037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670caff-1220-429c-9961-fc9a962e44d9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:40.757288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:40.836649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670caff-1220-429c-9961-fc9a962e44d9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:40.871952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1117654-d97b-4dca-8b33-a13db04e640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:40.872482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1117654-d97b-4dca-8b33-a13db04e640c None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:40.872758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1117654-d97b-4dca-8b33-a13db04e640c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:40.873026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1117654-d97b-4dca-8b33-a13db04e640c None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:40.873708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1117654-d97b-4dca-8b33-a13db04e640c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:40.875065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1117654-d97b-4dca-8b33-a13db04e640c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:40.889031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 655/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 10 13:28:40 2022] POST /identity/v3/services => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:28:42.849151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2231a-3fba-4589-be11-ea4ac7c48ae3 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.849736 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2231a-3fba-4589-be11-ea4ac7c48ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.849736 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2231a-3fba-4589-be11-ea4ac7c48ae3 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.851159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 641/1296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:42.856324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce484-e897-4352-ba89-58484ed51bf0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.856759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce484-e897-4352-ba89-58484ed51bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.856858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce484-e897-4352-ba89-58484ed51bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.874565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1bbce484-e897-4352-ba89-58484ed51bf0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:42.876258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1bbce484-e897-4352-ba89-58484ed51bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:42.946055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 656/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:42.968014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2231a-3fba-4589-be11-ea4ac7c48ae3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:43.065724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2231a-3fba-4589-be11-ea4ac7c48ae3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:43.099101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:43.099759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:43.099987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:43.100253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:43.100821 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:43.102197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:43.110765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:43.111910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:43.112768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 642/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:28:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:43.117660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbce484-e897-4352-ba89-58484ed51bf0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:43.135179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:43.213045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbce484-e897-4352-ba89-58484ed51bf0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:43.245470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:43.246073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:43.246347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:43.246616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:43.247244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:43.247542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:43.248904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:43.254715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 657/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 10 13:28:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:43.260547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:43.294379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:43.361621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1dcc1-6a5a-48ad-976c-2ee23c82f0f0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:43.393930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eef694-ec86-4abf-a70d-97f6bca374d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:43.394529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eef694-ec86-4abf-a70d-97f6bca374d3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:43.394803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eef694-ec86-4abf-a70d-97f6bca374d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:43.395069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eef694-ec86-4abf-a70d-97f6bca374d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:43.395751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eef694-ec86-4abf-a70d-97f6bca374d3 None admin] RBAC: Adding query filter params (interface=public, service_id=ac78f7a31cd245b9b28b1a728f46f9f1, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:43.396051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eef694-ec86-4abf-a70d-97f6bca374d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:43.397425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eef694-ec86-4abf-a70d-97f6bca374d3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:43.402231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 643/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 10 13:28:43 2022] GET /identity/v3/endpoints?service_id=ac78f7a31cd245b9b28b1a728f46f9f1&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:43.407959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:43.486371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:43.504534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965ccaa-7d82-4e71-bf5e-b1f507a686bb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:43.536966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239f972-b907-4ce8-b808-c056d541b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:43.537584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f972-b907-4ce8-b808-c056d541b064 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:43.537862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f972-b907-4ce8-b808-c056d541b064 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:43.538129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f972-b907-4ce8-b808-c056d541b064 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:43.538720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239f972-b907-4ce8-b808-c056d541b064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:43.540067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239f972-b907-4ce8-b808-c056d541b064 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:43.547965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 658/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 10 13:28:43 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:28:45.579266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49e01-4aa3-49ef-b979-35103a33eade None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:45.579788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49e01-4aa3-49ef-b979-35103a33eade None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:45.579788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49e01-4aa3-49ef-b979-35103a33eade None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:45.581045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 644/1302] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:28:45.586109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556127-2240-4c76-8240-d94831122925 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:45.586493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556127-2240-4c76-8240-d94831122925 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:45.586643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556127-2240-4c76-8240-d94831122925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:45.604530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ae556127-2240-4c76-8240-d94831122925 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:45.605725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ae556127-2240-4c76-8240-d94831122925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:45.669098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 659/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:28:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 13:28:45.692284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a49e01-4aa3-49ef-b979-35103a33eade None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:45.783362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:45.791044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a49e01-4aa3-49ef-b979-35103a33eade None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:45.826541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:45.827163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:45.827435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:45.827701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:45.828953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:45.830409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 13:28:45.838626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 13:28:45.839792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Aug 10 13:28:45.840886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 645/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 10 13:28:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 13:28:45.845766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556127-2240-4c76-8240-d94831122925 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:45.919002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:45.942538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556127-2240-4c76-8240-d94831122925 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:45.982288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ded80-8e62-4da0-8706-a939d2c4e9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:45.983146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ded80-8e62-4da0-8706-a939d2c4e9cb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:45.983420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ded80-8e62-4da0-8706-a939d2c4e9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:45.983685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ded80-8e62-4da0-8706-a939d2c4e9cb None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:45.984299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ded80-8e62-4da0-8706-a939d2c4e9cb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:28:45.985115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ded80-8e62-4da0-8706-a939d2c4e9cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:45.986492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ded80-8e62-4da0-8706-a939d2c4e9cb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:45.991564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 660/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 10 13:28:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:28:45.997151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:46.092243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44280e-d3cd-4c93-9c63-dd40e0ff6ed3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:46.124442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a9f95-1bca-4f15-b174-db28149613d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:46.125609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9f95-1bca-4f15-b174-db28149613d0 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:46.125901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9f95-1bca-4f15-b174-db28149613d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:46.126170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9f95-1bca-4f15-b174-db28149613d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:46.126812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a9f95-1bca-4f15-b174-db28149613d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:46.128158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a9f95-1bca-4f15-b174-db28149613d0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:46.146663 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 646/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:28:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:28:55.988529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=30 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:56.142469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:49.113600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8522f2-884a-42ab-9a7b-be7c1bca8e51 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:49.114079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8522f2-884a-42ab-9a7b-be7c1bca8e51 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:49.114160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8522f2-884a-42ab-9a7b-be7c1bca8e51 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:49.120653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba27c-f8bf-44eb-a4de-f89574a8de10 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:49.121067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba27c-f8bf-44eb-a4de-f89574a8de10 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:49.121207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba27c-f8bf-44eb-a4de-f89574a8de10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:49.126498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 661/1307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:29:49 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:29:49.282449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-749ba27c-f8bf-44eb-a4de-f89574a8de10 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:49.283897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-749ba27c-f8bf-44eb-a4de-f89574a8de10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:49.457934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 647/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:29:49.513252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30600216-cc8b-4275-b66d-77f122866943 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:49.513624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30600216-cc8b-4275-b66d-77f122866943 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:49.514383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30600216-cc8b-4275-b66d-77f122866943 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:49.515992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 662/1309] 10.222.0.52 () {54 vars in 965 bytes} [Wed Aug 10 13:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:29:49.523940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba541bd6-cf61-4198-a896-35a263049dd2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:49.524296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba541bd6-cf61-4198-a896-35a263049dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:49.524606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba541bd6-cf61-4198-a896-35a263049dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:49.610822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ba541bd6-cf61-4198-a896-35a263049dd2 None None] MFA Rules not processed for user `234eed198ab84e95ae1a7f03785a1481`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:49.612047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ba541bd6-cf61-4198-a896-35a263049dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:49.633274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:49.694355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 648/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Wed Aug 10 13:29:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:49.703756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30600216-cc8b-4275-b66d-77f122866943 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:49.705533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-30600216-cc8b-4275-b66d-77f122866943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:49.826877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30600216-cc8b-4275-b66d-77f122866943 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:49.911867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a01c03-a109-45a1-8893-1739b932cb0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:49.912488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a01c03-a109-45a1-8893-1739b932cb0d service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:49.912771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a01c03-a109-45a1-8893-1739b932cb0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:49.913058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a01c03-a109-45a1-8893-1739b932cb0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:49.914916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f3a01c03-a109-45a1-8893-1739b932cb0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:49.975671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:50.036642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a01c03-a109-45a1-8893-1739b932cb0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:50.038107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a01c03-a109-45a1-8893-1739b932cb0d service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:50.059422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 663/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:29:52.394788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b88a4-d2ec-46ba-9eed-3b359b3d3249 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:52.395436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b88a4-d2ec-46ba-9eed-3b359b3d3249 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:52.395436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b88a4-d2ec-46ba-9eed-3b359b3d3249 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:52.397055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 649/1312] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:29:52.402906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1cf3f-c3f5-45d5-b1a4-b4e25771600a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:52.403476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1cf3f-c3f5-45d5-b1a4-b4e25771600a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:52.403476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1cf3f-c3f5-45d5-b1a4-b4e25771600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:52.489909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-47f1cf3f-c3f5-45d5-b1a4-b4e25771600a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:52.491132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-47f1cf3f-c3f5-45d5-b1a4-b4e25771600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:52.597435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 664/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:52.635936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b88a4-d2ec-46ba-9eed-3b359b3d3249 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:52.637587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f29b88a4-d2ec-46ba-9eed-3b359b3d3249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:52.749651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b88a4-d2ec-46ba-9eed-3b359b3d3249 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:52.822928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:52.826394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7d54-3e8f-4d22-acc2-35848c172b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:52.827185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7d54-3e8f-4d22-acc2-35848c172b2c service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:52.827627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7d54-3e8f-4d22-acc2-35848c172b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:52.827873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7d54-3e8f-4d22-acc2-35848c172b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:52.829744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fefc7d54-3e8f-4d22-acc2-35848c172b2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:52.936800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7d54-3e8f-4d22-acc2-35848c172b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:52.938249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7d54-3e8f-4d22-acc2-35848c172b2c service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:52.956395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 650/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:29:55.233786 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d43e20-8694-4712-94f7-97fe4102b8f3 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:55.234323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d43e20-8694-4712-94f7-97fe4102b8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:55.234323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d43e20-8694-4712-94f7-97fe4102b8f3 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:55.235681 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 665/1315] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:29:55.240886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80a12-895a-44c9-a312-e9a6545912bd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:55.241342 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80a12-895a-44c9-a312-e9a6545912bd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:55.241436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80a12-895a-44c9-a312-e9a6545912bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:55.309122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6bd80a12-895a-44c9-a312-e9a6545912bd None None] MFA Rules not processed for user `45a4a1663ad84ad1953e68ba332fad3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:55.311056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6bd80a12-895a-44c9-a312-e9a6545912bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:55.379110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:55.425547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 651/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:29:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:55.464621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d43e20-8694-4712-94f7-97fe4102b8f3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:55.466174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-44d43e20-8694-4712-94f7-97fe4102b8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:55.476307 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:55.567057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d43e20-8694-4712-94f7-97fe4102b8f3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:55.636827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd3f70-8251-4589-829c-6986f5118221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:55.637440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd3f70-8251-4589-829c-6986f5118221 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:55.637718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd3f70-8251-4589-829c-6986f5118221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:55.637985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd3f70-8251-4589-829c-6986f5118221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:55.639576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-93cd3f70-8251-4589-829c-6986f5118221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:55.712713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:55.761901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd3f70-8251-4589-829c-6986f5118221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:55.763408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd3f70-8251-4589-829c-6986f5118221 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:55.781521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 666/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:29:58.352056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3e5ca-ab3e-4429-a87f-4260ae4d363d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:58.352607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3e5ca-ab3e-4429-a87f-4260ae4d363d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:58.352607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3e5ca-ab3e-4429-a87f-4260ae4d363d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:58.353955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 652/1318] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:29:58.359369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a294f3-cdb6-45cd-800b-8f0c0bd44f7e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:58.359785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a294f3-cdb6-45cd-800b-8f0c0bd44f7e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:58.359906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a294f3-cdb6-45cd-800b-8f0c0bd44f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:58.434086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-96a294f3-cdb6-45cd-800b-8f0c0bd44f7e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:29:58.436726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-96a294f3-cdb6-45cd-800b-8f0c0bd44f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:58.536091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 667/1319] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:29:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:29:58.574887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3e5ca-ab3e-4429-a87f-4260ae4d363d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:29:58.577069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-04b3e5ca-ab3e-4429-a87f-4260ae4d363d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:58.681451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3e5ca-ab3e-4429-a87f-4260ae4d363d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:29:58.752268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd3f46-b955-43e4-8026-df725445b522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:29:58.752865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd3f46-b955-43e4-8026-df725445b522 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:29:58.753137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd3f46-b955-43e4-8026-df725445b522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:29:58.753426 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd3f46-b955-43e4-8026-df725445b522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:29:58.754995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5edd3f46-b955-43e4-8026-df725445b522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:29:58.762121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:29:58.864452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd3f46-b955-43e4-8026-df725445b522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:29:58.865898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd3f46-b955-43e4-8026-df725445b522 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:29:58.884122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 653/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:29:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:01.619693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bab5-6183-4214-9bcf-604c070feda2 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:01.620209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bab5-6183-4214-9bcf-604c070feda2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:01.620209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bab5-6183-4214-9bcf-604c070feda2 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:01.621605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 668/1321] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:01.627249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f50d4-cb1a-4a34-aaa1-50780620ce0b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:01.627640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f50d4-cb1a-4a34-aaa1-50780620ce0b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:01.627785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f50d4-cb1a-4a34-aaa1-50780620ce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:01.693323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a87f50d4-cb1a-4a34-aaa1-50780620ce0b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:01.694522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a87f50d4-cb1a-4a34-aaa1-50780620ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:01.760152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:01.803351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 654/1322] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:01.855961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd5fda-9419-4455-9a54-1493111650be None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:01.856240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd5fda-9419-4455-9a54-1493111650be None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:01.856508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd5fda-9419-4455-9a54-1493111650be None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:01.858817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 669/1323] 10.222.0.52 () {54 vars in 965 bytes} [Wed Aug 10 13:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:01.865196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad444cb-0d89-4cd0-91c1-0e79049412ff None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:01.865601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad444cb-0d89-4cd0-91c1-0e79049412ff None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:01.865938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad444cb-0d89-4cd0-91c1-0e79049412ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:01.947429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7ad444cb-0d89-4cd0-91c1-0e79049412ff None None] MFA Rules not processed for user `234eed198ab84e95ae1a7f03785a1481`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:01.948542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7ad444cb-0d89-4cd0-91c1-0e79049412ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:02.027776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 655/1324] 10.222.0.52 () {58 vars in 1062 bytes} [Wed Aug 10 13:30:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:02.037083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd5fda-9419-4455-9a54-1493111650be None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:02.038624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-83cd5fda-9419-4455-9a54-1493111650be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:02.059153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:02.142888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd5fda-9419-4455-9a54-1493111650be None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:02.218377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536970ce-dccc-48f3-bcea-6a512c6478a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:02.218877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536970ce-dccc-48f3-bcea-6a512c6478a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:02.219151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536970ce-dccc-48f3-bcea-6a512c6478a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:02.219417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536970ce-dccc-48f3-bcea-6a512c6478a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:02.221002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-536970ce-dccc-48f3-bcea-6a512c6478a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:02.304270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:02.343820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536970ce-dccc-48f3-bcea-6a512c6478a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:02.345293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536970ce-dccc-48f3-bcea-6a512c6478a2 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:02.363355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 670/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:04.640984 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558c575-d99a-43db-a296-10f76956df7e None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:04.641528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558c575-d99a-43db-a296-10f76956df7e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:04.641646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558c575-d99a-43db-a296-10f76956df7e None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:04.643075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 656/1326] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:04.648076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df33ba-de61-43ee-80a0-dc715fa2f82f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:04.648522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df33ba-de61-43ee-80a0-dc715fa2f82f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:04.648751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df33ba-de61-43ee-80a0-dc715fa2f82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:04.737553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-34df33ba-de61-43ee-80a0-dc715fa2f82f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:04.738768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-34df33ba-de61-43ee-80a0-dc715fa2f82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:04.840152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 671/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:04.878205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558c575-d99a-43db-a296-10f76956df7e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:04.879747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3558c575-d99a-43db-a296-10f76956df7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:04.940082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:04.983717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558c575-d99a-43db-a296-10f76956df7e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:05.056445 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66fef5-2216-400b-b136-1b695263d5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:05.057058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66fef5-2216-400b-b136-1b695263d5fe service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:05.057362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66fef5-2216-400b-b136-1b695263d5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:05.057630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66fef5-2216-400b-b136-1b695263d5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:05.059233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6b66fef5-2216-400b-b136-1b695263d5fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:05.176920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:05.179293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66fef5-2216-400b-b136-1b695263d5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:05.180758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66fef5-2216-400b-b136-1b695263d5fe service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:05.200030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 657/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:07.284549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01348d5-ecfc-4748-96b1-fea42da72791 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:07.284991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01348d5-ecfc-4748-96b1-fea42da72791 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:07.285102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01348d5-ecfc-4748-96b1-fea42da72791 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:07.286410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 672/1329] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:07.291493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61017b8d-b8c3-4515-81ed-b9d91d486b73 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:07.291939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61017b8d-b8c3-4515-81ed-b9d91d486b73 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:07.292189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61017b8d-b8c3-4515-81ed-b9d91d486b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:07.361477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-61017b8d-b8c3-4515-81ed-b9d91d486b73 None None] MFA Rules not processed for user `45a4a1663ad84ad1953e68ba332fad3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:07.362775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-61017b8d-b8c3-4515-81ed-b9d91d486b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:07.479871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 658/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:07.521580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01348d5-ecfc-4748-96b1-fea42da72791 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:07.523145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c01348d5-ecfc-4748-96b1-fea42da72791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:07.557530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:07.640355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01348d5-ecfc-4748-96b1-fea42da72791 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:07.712955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eddc1-d0b5-449b-954f-4f7c6ab48252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:07.713592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eddc1-d0b5-449b-954f-4f7c6ab48252 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:07.713866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eddc1-d0b5-449b-954f-4f7c6ab48252 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:07.714134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eddc1-d0b5-449b-954f-4f7c6ab48252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:07.715744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0b7eddc1-d0b5-449b-954f-4f7c6ab48252 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:07.797647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:07.830321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eddc1-d0b5-449b-954f-4f7c6ab48252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:07.831682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eddc1-d0b5-449b-954f-4f7c6ab48252 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:07.849295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 673/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:10.095533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a5ff-6438-4c3e-b867-8a065fdd4eee None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:10.096090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a5ff-6438-4c3e-b867-8a065fdd4eee None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:10.096090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a5ff-6438-4c3e-b867-8a065fdd4eee None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:10.097436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 659/1332] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:10.102457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d6a9-1f46-4768-9616-6e4768b7d7d8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:10.102882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d6a9-1f46-4768-9616-6e4768b7d7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:10.102981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d6a9-1f46-4768-9616-6e4768b7d7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:10.168095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fb15d6a9-1f46-4768-9616-6e4768b7d7d8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:10.169729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fb15d6a9-1f46-4768-9616-6e4768b7d7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:10.263172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 674/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:10.300794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14a5ff-6438-4c3e-b867-8a065fdd4eee None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:10.302362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0e14a5ff-6438-4c3e-b867-8a065fdd4eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:10.370648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:10.404960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14a5ff-6438-4c3e-b867-8a065fdd4eee None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:10.476517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5970fd-a5e0-4097-899a-ed9d444f743f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:10.477051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5970fd-a5e0-4097-899a-ed9d444f743f service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:10.477358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5970fd-a5e0-4097-899a-ed9d444f743f service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:10.477628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5970fd-a5e0-4097-899a-ed9d444f743f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:10.479205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ff5970fd-a5e0-4097-899a-ed9d444f743f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:10.585512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5970fd-a5e0-4097-899a-ed9d444f743f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:10.586942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5970fd-a5e0-4097-899a-ed9d444f743f service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:10.597054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:10.605304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 660/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:12.668269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd994d-807d-4bf4-9338-e486256be167 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:12.668810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd994d-807d-4bf4-9338-e486256be167 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:12.668810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd994d-807d-4bf4-9338-e486256be167 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:12.670801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 675/1335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:12.676007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147ce7-8228-4b81-b744-4be628ca564d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:12.676436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147ce7-8228-4b81-b744-4be628ca564d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:12.676543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147ce7-8228-4b81-b744-4be628ca564d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:12.740797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fc147ce7-8228-4b81-b744-4be628ca564d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:12.742016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fc147ce7-8228-4b81-b744-4be628ca564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:12.836850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 661/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:12.877585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd994d-807d-4bf4-9338-e486256be167 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:12.879116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b1bd994d-807d-4bf4-9338-e486256be167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:12.920891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:12.984280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd994d-807d-4bf4-9338-e486256be167 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:13.054816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e231578-0441-44d8-b6cf-ae778554af33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:13.055406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e231578-0441-44d8-b6cf-ae778554af33 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:13.055676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e231578-0441-44d8-b6cf-ae778554af33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:13.055936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e231578-0441-44d8-b6cf-ae778554af33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:13.057525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9e231578-0441-44d8-b6cf-ae778554af33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:13.138628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:13.163976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e231578-0441-44d8-b6cf-ae778554af33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:13.165466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e231578-0441-44d8-b6cf-ae778554af33 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:13.183550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 676/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:15.819869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8687-3635-439d-a4e5-e17466131080 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:15.820432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8687-3635-439d-a4e5-e17466131080 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:15.820432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8687-3635-439d-a4e5-e17466131080 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:15.821788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 662/1338] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:15.827445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6dea-5f1e-44b8-be48-4610588ff240 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:15.827893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6dea-5f1e-44b8-be48-4610588ff240 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:15.828009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6dea-5f1e-44b8-be48-4610588ff240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:15.893581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4afc6dea-5f1e-44b8-be48-4610588ff240 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:15.894768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4afc6dea-5f1e-44b8-be48-4610588ff240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:15.987883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 677/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:30:16.026330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8687-3635-439d-a4e5-e17466131080 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:16.027888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-faad8687-3635-439d-a4e5-e17466131080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:16.101822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:16.130319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8687-3635-439d-a4e5-e17466131080 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:16.201085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c91c32-c895-4722-85ee-52ee1633385f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:16.201722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c91c32-c895-4722-85ee-52ee1633385f service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:16.202000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c91c32-c895-4722-85ee-52ee1633385f service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:16.202268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c91c32-c895-4722-85ee-52ee1633385f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:16.203846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-00c91c32-c895-4722-85ee-52ee1633385f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:16.310097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c91c32-c895-4722-85ee-52ee1633385f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:16.311533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c91c32-c895-4722-85ee-52ee1633385f service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:16.331248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 663/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:20.389805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9dcd-3d1c-437c-8bf2-fee69b564728 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:20.390376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9dcd-3d1c-437c-8bf2-fee69b564728 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:20.390376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9dcd-3d1c-437c-8bf2-fee69b564728 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:20.391700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 678/1341] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:20.396903 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773069f2-bdbf-4585-8dc2-3459c51555ab None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:20.397377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773069f2-bdbf-4585-8dc2-3459c51555ab None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:20.397472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773069f2-bdbf-4585-8dc2-3459c51555ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:20.402994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:20.466452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-773069f2-bdbf-4585-8dc2-3459c51555ab None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:20.467779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-773069f2-bdbf-4585-8dc2-3459c51555ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:20.581967 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 664/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:20.621541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e9dcd-3d1c-437c-8bf2-fee69b564728 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:20.623609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e10e9dcd-3d1c-437c-8bf2-fee69b564728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:20.672235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:20.727498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e9dcd-3d1c-437c-8bf2-fee69b564728 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:20.804365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5517bf8-2ef4-43ce-b2e2-784ca67b8390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:20.804965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517bf8-2ef4-43ce-b2e2-784ca67b8390 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:20.805253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517bf8-2ef4-43ce-b2e2-784ca67b8390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:20.805533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517bf8-2ef4-43ce-b2e2-784ca67b8390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:20.807125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f5517bf8-2ef4-43ce-b2e2-784ca67b8390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:20.910589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:20.927934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5517bf8-2ef4-43ce-b2e2-784ca67b8390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:20.929404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5517bf8-2ef4-43ce-b2e2-784ca67b8390 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:20.946933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 679/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:23.584617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e361f-d400-4553-bb9d-6fd59b1ead53 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:23.585174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e361f-d400-4553-bb9d-6fd59b1ead53 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:23.585174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e361f-d400-4553-bb9d-6fd59b1ead53 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:23.586482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 665/1344] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:23.592674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48c81f-1a85-4304-841a-c662e807465d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:23.593181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48c81f-1a85-4304-841a-c662e807465d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:23.593181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48c81f-1a85-4304-841a-c662e807465d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:23.665197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-da48c81f-1a85-4304-841a-c662e807465d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:23.666452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-da48c81f-1a85-4304-841a-c662e807465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:23.766368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 680/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:23.804082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e361f-d400-4553-bb9d-6fd59b1ead53 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:23.805788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-886e361f-d400-4553-bb9d-6fd59b1ead53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:23.895646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:23.912803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e361f-d400-4553-bb9d-6fd59b1ead53 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:23.986214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5772877-7fea-45a4-9f79-d26aaacfbc05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:23.986733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5772877-7fea-45a4-9f79-d26aaacfbc05 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:23.987009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5772877-7fea-45a4-9f79-d26aaacfbc05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:23.987280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5772877-7fea-45a4-9f79-d26aaacfbc05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:23.988857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d5772877-7fea-45a4-9f79-d26aaacfbc05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:24.097089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5772877-7fea-45a4-9f79-d26aaacfbc05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:24.098560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5772877-7fea-45a4-9f79-d26aaacfbc05 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:24.117005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 666/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:28.076248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1f7d-8459-4712-a8c8-1cb2fa00b1af None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:28.076772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1f7d-8459-4712-a8c8-1cb2fa00b1af None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:28.076772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1f7d-8459-4712-a8c8-1cb2fa00b1af None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:28.081328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 681/1347] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:28.085504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf1c05-f534-4f8d-8c5d-2846c5c91535 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:28.085964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf1c05-f534-4f8d-8c5d-2846c5c91535 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:28.086270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf1c05-f534-4f8d-8c5d-2846c5c91535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:28.105366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:28.305650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bddf1c05-f534-4f8d-8c5d-2846c5c91535 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:28.307024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bddf1c05-f534-4f8d-8c5d-2846c5c91535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:28.423940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 667/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:28.462224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1f7d-8459-4712-a8c8-1cb2fa00b1af None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:28.463887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c69a1f7d-8459-4712-a8c8-1cb2fa00b1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:28.555997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:28.643479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1f7d-8459-4712-a8c8-1cb2fa00b1af None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:28.736157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a1799-a223-4675-95f3-1c983d965e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:28.736702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a1799-a223-4675-95f3-1c983d965e1d service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:28.736981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a1799-a223-4675-95f3-1c983d965e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:28.737335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a1799-a223-4675-95f3-1c983d965e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:28.738973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-943a1799-a223-4675-95f3-1c983d965e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:28.889621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:28.892161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a1799-a223-4675-95f3-1c983d965e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:28.894810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a1799-a223-4675-95f3-1c983d965e1d service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:28.921503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 682/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:34.361488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd3447-a132-492c-94c0-7779606d560a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:34.362417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd3447-a132-492c-94c0-7779606d560a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:34.362920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd3447-a132-492c-94c0-7779606d560a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:34.365707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 668/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:30:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:34.373710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1641c2-3bc5-40d2-991f-26545e4134d4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:34.374164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1641c2-3bc5-40d2-991f-26545e4134d4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:34.374244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1641c2-3bc5-40d2-991f-26545e4134d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:34.568227 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3d1641c2-3bc5-40d2-991f-26545e4134d4 None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:34.570552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3d1641c2-3bc5-40d2-991f-26545e4134d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:34.723882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 683/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:30:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:38.421894 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=59 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:39.642141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6139-acbc-4b39-a177-a1a61ea02502 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:39.643243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6139-acbc-4b39-a177-a1a61ea02502 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:39.643677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6139-acbc-4b39-a177-a1a61ea02502 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:39.645088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 669/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:39.650092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cab54-3074-49ef-ae96-293caa3adf47 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:39.650530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cab54-3074-49ef-ae96-293caa3adf47 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:39.650895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cab54-3074-49ef-ae96-293caa3adf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:39.705624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:39.733520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d00cab54-3074-49ef-ae96-293caa3adf47 None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:39.734794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d00cab54-3074-49ef-ae96-293caa3adf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:39.863523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 684/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:30:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:41.026296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc76e0a-e237-4316-80ff-8da435fcfc92 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:41.026814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc76e0a-e237-4316-80ff-8da435fcfc92 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:41.026814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc76e0a-e237-4316-80ff-8da435fcfc92 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:41.028696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 670/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:41.033652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25322c-f19c-400d-9cd2-c2d385df7b4f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:41.034096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25322c-f19c-400d-9cd2-c2d385df7b4f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:41.034330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25322c-f19c-400d-9cd2-c2d385df7b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:41.112648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9f25322c-f19c-400d-9cd2-c2d385df7b4f None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:41.113806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9f25322c-f19c-400d-9cd2-c2d385df7b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:41.155223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:41.219717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 685/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:30:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:30:44.617028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b371-5ce1-4258-9dcc-a2597b4ffe9f None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:44.618042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b371-5ce1-4258-9dcc-a2597b4ffe9f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:44.618523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92b371-5ce1-4258-9dcc-a2597b4ffe9f None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:44.620337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 671/1356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:44.625873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0249f5e-24b4-43ab-bba4-0ef19c5ba4c7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:44.628081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0249f5e-24b4-43ab-bba4-0ef19c5ba4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:44.628081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0249f5e-24b4-43ab-bba4-0ef19c5ba4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:44.704135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e0249f5e-24b4-43ab-bba4-0ef19c5ba4c7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:44.705410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e0249f5e-24b4-43ab-bba4-0ef19c5ba4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:44.809482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 686/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:44.864613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a822cd-f50e-4516-bb1e-a2924916ef2f None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:44.865171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a822cd-f50e-4516-bb1e-a2924916ef2f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:44.865745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a822cd-f50e-4516-bb1e-a2924916ef2f None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:44.867767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 672/1358] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 10 13:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:44.878105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7d4a0-8bec-41e1-bee2-9e443645b90a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:44.878788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7d4a0-8bec-41e1-bee2-9e443645b90a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:44.879262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7d4a0-8bec-41e1-bee2-9e443645b90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:44.900407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:44.964257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-23a7d4a0-8bec-41e1-bee2-9e443645b90a None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:44.965847 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-23a7d4a0-8bec-41e1-bee2-9e443645b90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:45.090214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 687/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 10 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:30:45.098631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a822cd-f50e-4516-bb1e-a2924916ef2f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:45.100309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-21a822cd-f50e-4516-bb1e-a2924916ef2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:45.264823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a822cd-f50e-4516-bb1e-a2924916ef2f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:45.347719 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:45.400833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b77de-c0f6-46a9-b44f-d2290fef98fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:45.401996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b77de-c0f6-46a9-b44f-d2290fef98fb service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:45.403069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b77de-c0f6-46a9-b44f-d2290fef98fb service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:45.403537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b77de-c0f6-46a9-b44f-d2290fef98fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:45.405624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8b1b77de-c0f6-46a9-b44f-d2290fef98fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:45.506525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b77de-c0f6-46a9-b44f-d2290fef98fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:45.508478 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b77de-c0f6-46a9-b44f-d2290fef98fb service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:45.528577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 673/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:47.769294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3428ad-1f88-4d04-aac5-33d439a91a53 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:47.769862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3428ad-1f88-4d04-aac5-33d439a91a53 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:47.769862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3428ad-1f88-4d04-aac5-33d439a91a53 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:47.771446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 688/1361] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:47.776524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0938411-3bc2-439b-98ab-ad81cc2ccfe7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:47.777489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0938411-3bc2-439b-98ab-ad81cc2ccfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:47.777683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0938411-3bc2-439b-98ab-ad81cc2ccfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:47.847127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f0938411-3bc2-439b-98ab-ad81cc2ccfe7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:47.848309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f0938411-3bc2-439b-98ab-ad81cc2ccfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:47.891083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:47.969286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 674/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:48.018045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3428ad-1f88-4d04-aac5-33d439a91a53 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:48.020341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8d3428ad-1f88-4d04-aac5-33d439a91a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:48.098581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:48.195542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3428ad-1f88-4d04-aac5-33d439a91a53 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:48.331025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1153058-6941-43d7-9c99-74fab27aa67a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:48.331631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1153058-6941-43d7-9c99-74fab27aa67a service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:48.331904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1153058-6941-43d7-9c99-74fab27aa67a service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:48.332172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1153058-6941-43d7-9c99-74fab27aa67a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:48.334154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d1153058-6941-43d7-9c99-74fab27aa67a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:48.360433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:48.434834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1153058-6941-43d7-9c99-74fab27aa67a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:48.436858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1153058-6941-43d7-9c99-74fab27aa67a service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:48.456558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 689/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:50.828948 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974c88d-0ade-4ee0-9f75-fde213dcd0c5 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:50.829559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974c88d-0ade-4ee0-9f75-fde213dcd0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:50.829559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974c88d-0ade-4ee0-9f75-fde213dcd0c5 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:50.830858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 675/1364] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:50.836922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e221a9-749a-4486-b4c6-b7f63c1b5f4f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:50.837714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e221a9-749a-4486-b4c6-b7f63c1b5f4f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:50.838169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e221a9-749a-4486-b4c6-b7f63c1b5f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:50.913600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-35e221a9-749a-4486-b4c6-b7f63c1b5f4f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:50.914811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-35e221a9-749a-4486-b4c6-b7f63c1b5f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:51.017745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:51.024574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 690/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:51.064076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bfd9-e1f7-42fa-b7f8-93813a6bc206 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:51.064433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bfd9-e1f7-42fa-b7f8-93813a6bc206 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:51.064696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bfd9-e1f7-42fa-b7f8-93813a6bc206 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:51.066090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 676/1366] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 10 13:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:51.071876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5411a-98a9-4ed2-b547-d979cc0b6c4b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:51.072266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5411a-98a9-4ed2-b547-d979cc0b6c4b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:51.072749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5411a-98a9-4ed2-b547-d979cc0b6c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:51.162332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b8c5411a-98a9-4ed2-b547-d979cc0b6c4b None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:51.163785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b8c5411a-98a9-4ed2-b547-d979cc0b6c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:51.294528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 691/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 10 13:30:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:51.303216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8bfd9-e1f7-42fa-b7f8-93813a6bc206 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:51.304779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-97c8bfd9-e1f7-42fa-b7f8-93813a6bc206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:51.460249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8bfd9-e1f7-42fa-b7f8-93813a6bc206 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:51.482133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:51.579759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d13b86-36a9-478d-b390-4c1f4f31e3d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:51.580351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d13b86-36a9-478d-b390-4c1f4f31e3d0 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:51.580628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d13b86-36a9-478d-b390-4c1f4f31e3d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:51.580897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d13b86-36a9-478d-b390-4c1f4f31e3d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:51.582505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-97d13b86-36a9-478d-b390-4c1f4f31e3d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:51.687897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d13b86-36a9-478d-b390-4c1f4f31e3d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:51.689522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d13b86-36a9-478d-b390-4c1f4f31e3d0 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:51.708634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 677/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:52.697061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86f7e3-93c6-4544-8f45-d6fee377e81d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:52.697655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86f7e3-93c6-4544-8f45-d6fee377e81d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:52.697931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86f7e3-93c6-4544-8f45-d6fee377e81d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:52.699391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 692/1369] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:52.704335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bee27-304d-488f-80d7-1cdfef6fafcf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:52.704769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bee27-304d-488f-80d7-1cdfef6fafcf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:52.704862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bee27-304d-488f-80d7-1cdfef6fafcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:52.739474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:52.776589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-df7bee27-304d-488f-80d7-1cdfef6fafcf None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:52.777965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-df7bee27-304d-488f-80d7-1cdfef6fafcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:52.882250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 678/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:53.231465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3fb0-22b6-4c00-932b-d02d29d63db5 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:53.231465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3fb0-22b6-4c00-932b-d02d29d63db5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:53.231465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3fb0-22b6-4c00-932b-d02d29d63db5 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:53.231465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 693/1371] 10.222.0.52 () {54 vars in 964 bytes} [Wed Aug 10 13:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:53.236518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027cda-711c-47e6-96f3-eefbd54339b6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:53.236799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027cda-711c-47e6-96f3-eefbd54339b6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:53.237067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027cda-711c-47e6-96f3-eefbd54339b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:53.325813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-72027cda-711c-47e6-96f3-eefbd54339b6 None None] MFA Rules not processed for user `b96be241231247b39e544529a4cd0b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:53.327176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-72027cda-711c-47e6-96f3-eefbd54339b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:53.380385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:53.412717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 679/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Aug 10 13:30:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:53.420703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed3fb0-22b6-4c00-932b-d02d29d63db5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:53.422368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b7ed3fb0-22b6-4c00-932b-d02d29d63db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:53.476570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:53.533979 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed3fb0-22b6-4c00-932b-d02d29d63db5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:53.611781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b885e2-7322-4f36-8312-77ed39a7bc49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:53.612380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b885e2-7322-4f36-8312-77ed39a7bc49 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:53.612652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b885e2-7322-4f36-8312-77ed39a7bc49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:53.612921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b885e2-7322-4f36-8312-77ed39a7bc49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:53.614561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-04b885e2-7322-4f36-8312-77ed39a7bc49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:53.728867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:53.744296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b885e2-7322-4f36-8312-77ed39a7bc49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:53.745927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b885e2-7322-4f36-8312-77ed39a7bc49 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:53.767466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 694/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:53.797772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5efab-4928-4ea9-b6c6-2ab582344b8c None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:53.798504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5efab-4928-4ea9-b6c6-2ab582344b8c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:53.798504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5efab-4928-4ea9-b6c6-2ab582344b8c None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:53.803566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 680/1374] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:53.805637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11af90-498a-461a-bc2f-74fc86c48ec3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:53.805960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11af90-498a-461a-bc2f-74fc86c48ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:53.806346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11af90-498a-461a-bc2f-74fc86c48ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:53.875524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1f11af90-498a-461a-bc2f-74fc86c48ec3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:53.876718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1f11af90-498a-461a-bc2f-74fc86c48ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:53.977854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 695/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:54.020194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5efab-4928-4ea9-b6c6-2ab582344b8c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:54.021888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-98f5efab-4928-4ea9-b6c6-2ab582344b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:54.188367 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5efab-4928-4ea9-b6c6-2ab582344b8c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:54.242660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:54.307874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076114b-34e0-430b-ac75-c5e677e7776e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:54.309091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076114b-34e0-430b-ac75-c5e677e7776e service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.309401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076114b-34e0-430b-ac75-c5e677e7776e service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.309671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076114b-34e0-430b-ac75-c5e677e7776e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.311308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c076114b-34e0-430b-ac75-c5e677e7776e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:54.398227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076114b-34e0-430b-ac75-c5e677e7776e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:54.399816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076114b-34e0-430b-ac75-c5e677e7776e service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:54.418620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 681/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:56.445190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beecb1-6790-4ae6-b855-4307f343b00d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.445731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beecb1-6790-4ae6-b855-4307f343b00d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.445731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beecb1-6790-4ae6-b855-4307f343b00d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.447091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 696/1377] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:56.452266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac19664-326b-4453-aa93-642301e79b63 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.452699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac19664-326b-4453-aa93-642301e79b63 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.452806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac19664-326b-4453-aa93-642301e79b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.518217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5ac19664-326b-4453-aa93-642301e79b63 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:56.519396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5ac19664-326b-4453-aa93-642301e79b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:56.525841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:56.628851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 682/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:56.672861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13beecb1-6790-4ae6-b855-4307f343b00d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:56.674380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-13beecb1-6790-4ae6-b855-4307f343b00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:56.737513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:56.847016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13beecb1-6790-4ae6-b855-4307f343b00d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:56.959863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692455d2-6e66-4de9-952f-871c9fcb275e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:56.960461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692455d2-6e66-4de9-952f-871c9fcb275e service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:56.960734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692455d2-6e66-4de9-952f-871c9fcb275e service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:56.961000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692455d2-6e66-4de9-952f-871c9fcb275e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:56.962589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-692455d2-6e66-4de9-952f-871c9fcb275e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:56.982409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:57.057910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692455d2-6e66-4de9-952f-871c9fcb275e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:57.059506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692455d2-6e66-4de9-952f-871c9fcb275e service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:57.077163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 697/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:59.154288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9801f55-3046-4c34-9c27-e03e770d1196 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:59.154842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9801f55-3046-4c34-9c27-e03e770d1196 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:59.154842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9801f55-3046-4c34-9c27-e03e770d1196 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:59.156324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 683/1380] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:30:59.161395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337a64-651f-43e8-b8f0-3d306d42b97b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:59.161842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337a64-651f-43e8-b8f0-3d306d42b97b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:59.161937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337a64-651f-43e8-b8f0-3d306d42b97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:59.230836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f2337a64-651f-43e8-b8f0-3d306d42b97b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:30:59.232403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f2337a64-651f-43e8-b8f0-3d306d42b97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:59.344115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:59.355559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 698/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:30:59.396944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9801f55-3046-4c34-9c27-e03e770d1196 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:59.398689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e9801f55-3046-4c34-9c27-e03e770d1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:59.540236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:59.552673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9801f55-3046-4c34-9c27-e03e770d1196 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:59.683397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b89ff-ff21-4169-a324-2c7c1506336e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:59.684008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b89ff-ff21-4169-a324-2c7c1506336e service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:59.684278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b89ff-ff21-4169-a324-2c7c1506336e service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:59.684540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b89ff-ff21-4169-a324-2c7c1506336e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:59.686135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-344b89ff-ff21-4169-a324-2c7c1506336e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:59.780437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b89ff-ff21-4169-a324-2c7c1506336e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:59.782043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b89ff-ff21-4169-a324-2c7c1506336e service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:59.793722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:59.803291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 684/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:01.858091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace07a5-f769-4e67-86fb-6faa921e901f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:01.858668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace07a5-f769-4e67-86fb-6faa921e901f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:01.858803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace07a5-f769-4e67-86fb-6faa921e901f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:01.860646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 699/1383] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:31:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:01.866285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfde8e-f311-471d-9f61-9bc16018b4ad None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:01.866697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfde8e-f311-471d-9f61-9bc16018b4ad None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:01.866970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfde8e-f311-471d-9f61-9bc16018b4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:01.935068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e8dfde8e-f311-471d-9f61-9bc16018b4ad None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:01.936283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e8dfde8e-f311-471d-9f61-9bc16018b4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:02.032281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 685/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:02.076492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace07a5-f769-4e67-86fb-6faa921e901f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:02.078058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8ace07a5-f769-4e67-86fb-6faa921e901f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:02.241395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace07a5-f769-4e67-86fb-6faa921e901f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:02.312603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:02.360164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9becd82-76ac-474b-9df1-bae0df54ed8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:02.360776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9becd82-76ac-474b-9df1-bae0df54ed8b service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:02.361058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9becd82-76ac-474b-9df1-bae0df54ed8b service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:02.361377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9becd82-76ac-474b-9df1-bae0df54ed8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:02.362972 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b9becd82-76ac-474b-9df1-bae0df54ed8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:02.454382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9becd82-76ac-474b-9df1-bae0df54ed8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:02.456095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9becd82-76ac-474b-9df1-bae0df54ed8b service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:02.475220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 700/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:04.476524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bfda9-61bc-4417-9b9a-631fe2a1adfc None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:04.477218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bfda9-61bc-4417-9b9a-631fe2a1adfc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:04.477218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bfda9-61bc-4417-9b9a-631fe2a1adfc None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:04.479513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 686/1386] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:31:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:04.486798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190e687-b439-4f3c-83fa-8bec8bd0bf51 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:04.487254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190e687-b439-4f3c-83fa-8bec8bd0bf51 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:04.487485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190e687-b439-4f3c-83fa-8bec8bd0bf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:04.569190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3190e687-b439-4f3c-83fa-8bec8bd0bf51 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:04.570806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3190e687-b439-4f3c-83fa-8bec8bd0bf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:04.612223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:04.683537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 701/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:04.728828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bfda9-61bc-4417-9b9a-631fe2a1adfc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:04.731166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b49bfda9-61bc-4417-9b9a-631fe2a1adfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:04.820896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:04.898974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bfda9-61bc-4417-9b9a-631fe2a1adfc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:05.019600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97080bfb-2627-48ac-8c24-5024420ef5df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:05.020118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080bfb-2627-48ac-8c24-5024420ef5df service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:05.020394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080bfb-2627-48ac-8c24-5024420ef5df service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:05.020665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080bfb-2627-48ac-8c24-5024420ef5df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:05.022290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-97080bfb-2627-48ac-8c24-5024420ef5df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:05.066618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:05.122656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97080bfb-2627-48ac-8c24-5024420ef5df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:05.124234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97080bfb-2627-48ac-8c24-5024420ef5df service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:05.142380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 687/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:07.254115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3b934-9d73-4e5e-8ef5-d1820f8d0c60 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:07.254678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3b934-9d73-4e5e-8ef5-d1820f8d0c60 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:07.254678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3b934-9d73-4e5e-8ef5-d1820f8d0c60 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:07.256222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 702/1389] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:07.261248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830f2da-b3bf-42f1-97a1-2d26a42e4cb8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:07.261671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830f2da-b3bf-42f1-97a1-2d26a42e4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:07.261806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830f2da-b3bf-42f1-97a1-2d26a42e4cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:07.331545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0830f2da-b3bf-42f1-97a1-2d26a42e4cb8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:07.332744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0830f2da-b3bf-42f1-97a1-2d26a42e4cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:07.430167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 688/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:07.473992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3b934-9d73-4e5e-8ef5-d1820f8d0c60 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:07.475621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-38b3b934-9d73-4e5e-8ef5-d1820f8d0c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:07.631016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3b934-9d73-4e5e-8ef5-d1820f8d0c60 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:07.647656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:07.748834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfcf41-966c-4286-bcf8-a542fb4f3d56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:07.749464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfcf41-966c-4286-bcf8-a542fb4f3d56 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:07.749741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfcf41-966c-4286-bcf8-a542fb4f3d56 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:07.750019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfcf41-966c-4286-bcf8-a542fb4f3d56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:07.751622 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d6cfcf41-966c-4286-bcf8-a542fb4f3d56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:07.840037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfcf41-966c-4286-bcf8-a542fb4f3d56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:07.841682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfcf41-966c-4286-bcf8-a542fb4f3d56 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:07.860581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 703/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:31:08.892984 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e6c2-93c6-4ab7-acdf-26f2d1942b18 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:08.893544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e6c2-93c6-4ab7-acdf-26f2d1942b18 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:08.893636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e6c2-93c6-4ab7-acdf-26f2d1942b18 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:08.894980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 689/1392] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:31:08.900089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919adf0a-e41d-41ba-bd45-5c217a9d7f78 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:08.900508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919adf0a-e41d-41ba-bd45-5c217a9d7f78 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:08.900613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919adf0a-e41d-41ba-bd45-5c217a9d7f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:08.926409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:08.969056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-919adf0a-e41d-41ba-bd45-5c217a9d7f78 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:08.970281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-919adf0a-e41d-41ba-bd45-5c217a9d7f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.071612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 704/1393] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:09.127937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307a63-7d20-4f9a-acf2-200c14b75f36 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.128219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307a63-7d20-4f9a-acf2-200c14b75f36 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.128490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307a63-7d20-4f9a-acf2-200c14b75f36 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.129867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 690/1394] 10.222.0.52 () {54 vars in 967 bytes} [Wed Aug 10 13:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:09.134815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb8e62-fd92-4f0e-b6e1-8c914f25ff50 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.135101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb8e62-fd92-4f0e-b6e1-8c914f25ff50 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.135374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb8e62-fd92-4f0e-b6e1-8c914f25ff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.217868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6afb8e62-fd92-4f0e-b6e1-8c914f25ff50 None None] MFA Rules not processed for user `ce851cf0509947e783dbd5b4809bc897`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:09.217868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6afb8e62-fd92-4f0e-b6e1-8c914f25ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.262797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:09.304029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 705/1395] 10.222.0.52 () {58 vars in 1064 bytes} [Wed Aug 10 13:31:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:09.312144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b307a63-7d20-4f9a-acf2-200c14b75f36 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:09.313670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9b307a63-7d20-4f9a-acf2-200c14b75f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.327281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:09.418422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b307a63-7d20-4f9a-acf2-200c14b75f36 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:09.491911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a93834-0105-4ce8-8b04-b454b763c76f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce851cf0509947e783dbd5b4809bc897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:09.492511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a93834-0105-4ce8-8b04-b454b763c76f service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.492784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a93834-0105-4ce8-8b04-b454b763c76f service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.493052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a93834-0105-4ce8-8b04-b454b763c76f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.494644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c4a93834-0105-4ce8-8b04-b454b763c76f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:09.563815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:09.607737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a93834-0105-4ce8-8b04-b454b763c76f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:09.609177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a93834-0105-4ce8-8b04-b454b763c76f service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:09.628815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 691/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:09.925783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a884469-8ef0-4a2a-a8ed-9b641f633c7b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.926402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a884469-8ef0-4a2a-a8ed-9b641f633c7b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.926402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a884469-8ef0-4a2a-a8ed-9b641f633c7b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:09.927670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 706/1397] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:09.933327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc09655-fd64-4b20-85ac-160ae3f60f98 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:09.933796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc09655-fd64-4b20-85ac-160ae3f60f98 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:09.933914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc09655-fd64-4b20-85ac-160ae3f60f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:10.005290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bfc09655-fd64-4b20-85ac-160ae3f60f98 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:10.005290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bfc09655-fd64-4b20-85ac-160ae3f60f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:10.102195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 692/1398] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:10.146556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a884469-8ef0-4a2a-a8ed-9b641f633c7b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:10.148071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2a884469-8ef0-4a2a-a8ed-9b641f633c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:10.313067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a884469-8ef0-4a2a-a8ed-9b641f633c7b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:10.363359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:10.363658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804da586-2d88-4867-b2c8-fbced69e3d97 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:10.363949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804da586-2d88-4867-b2c8-fbced69e3d97 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:10.364346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804da586-2d88-4867-b2c8-fbced69e3d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:10.391761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:10.431782 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-804da586-2d88-4867-b2c8-fbced69e3d97 None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:10.432292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e065538-643f-4ecb-adc9-c0756fcdb8b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:10.433420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065538-643f-4ecb-adc9-c0756fcdb8b2 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:10.433546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-804da586-2d88-4867-b2c8-fbced69e3d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:10.433688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065538-643f-4ecb-adc9-c0756fcdb8b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:10.433954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065538-643f-4ecb-adc9-c0756fcdb8b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:10.435524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8e065538-643f-4ecb-adc9-c0756fcdb8b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:10.526550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 693/1399] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:31:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:10.536892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e065538-643f-4ecb-adc9-c0756fcdb8b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:10.538650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e065538-643f-4ecb-adc9-c0756fcdb8b2 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:10.558974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 707/1400] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:10.570201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082a2f0-cdb4-4d55-9228-b66aafaa715f None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:10.570571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082a2f0-cdb4-4d55-9228-b66aafaa715f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:10.570950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082a2f0-cdb4-4d55-9228-b66aafaa715f None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:10.572540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 694/1401] 10.222.0.52 () {54 vars in 943 bytes} [Wed Aug 10 13:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:10.580328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664539-2b75-47a5-86df-2996b1b24316 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:10.580615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664539-2b75-47a5-86df-2996b1b24316 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:10.581021 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664539-2b75-47a5-86df-2996b1b24316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:10.648369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:10.664405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f4664539-2b75-47a5-86df-2996b1b24316 None None] MFA Rules not processed for user `10209fea1bcc43509534b7e20f635c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:10.665668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f4664539-2b75-47a5-86df-2996b1b24316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:10.794963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 708/1402] 10.222.0.52 () {58 vars in 1040 bytes} [Wed Aug 10 13:31:10 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:10.805052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082a2f0-cdb4-4d55-9228-b66aafaa715f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:10.806615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3082a2f0-cdb4-4d55-9228-b66aafaa715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:10.919118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:10.947852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082a2f0-cdb4-4d55-9228-b66aafaa715f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:11.051460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e491b24-5ca6-419a-8dee-d81f913d3b0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:11.052061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e491b24-5ca6-419a-8dee-d81f913d3b0e service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:11.052329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e491b24-5ca6-419a-8dee-d81f913d3b0e service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:11.052589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e491b24-5ca6-419a-8dee-d81f913d3b0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:11.054230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1e491b24-5ca6-419a-8dee-d81f913d3b0e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:11.135957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e491b24-5ca6-419a-8dee-d81f913d3b0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:11.137580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e491b24-5ca6-419a-8dee-d81f913d3b0e service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:11.145716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 695/1403] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:31:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:31:12.628678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57668cdb-b733-4f9a-b765-847e2538809b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.631061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57668cdb-b733-4f9a-b765-847e2538809b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.631061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57668cdb-b733-4f9a-b765-847e2538809b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.631061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 709/1404] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:12.636551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf60e8-c8a1-48a9-8e9c-ef350deaad16 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:12.637009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf60e8-c8a1-48a9-8e9c-ef350deaad16 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:12.637117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf60e8-c8a1-48a9-8e9c-ef350deaad16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:12.652145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:12.712337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ebaf60e8-c8a1-48a9-8e9c-ef350deaad16 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:12.713709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ebaf60e8-c8a1-48a9-8e9c-ef350deaad16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:12.863234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 696/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:12.912492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57668cdb-b733-4f9a-b765-847e2538809b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:12.914047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-57668cdb-b733-4f9a-b765-847e2538809b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:13.042378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:13.091752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57668cdb-b733-4f9a-b765-847e2538809b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:13.235719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd77e10-336e-47e9-9e86-08e15375f749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:13.236241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd77e10-336e-47e9-9e86-08e15375f749 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:13.236518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd77e10-336e-47e9-9e86-08e15375f749 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:13.236817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd77e10-336e-47e9-9e86-08e15375f749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:13.238554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ccd77e10-336e-47e9-9e86-08e15375f749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:13.339097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:13.357750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd77e10-336e-47e9-9e86-08e15375f749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:13.359365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd77e10-336e-47e9-9e86-08e15375f749 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:13.385645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 710/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:15.440871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd1f58-2e21-4e80-8441-377f4e76ebfc None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:15.441468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd1f58-2e21-4e80-8441-377f4e76ebfc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:15.441468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd1f58-2e21-4e80-8441-377f4e76ebfc None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:15.442778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 697/1407] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:15.447902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bb98f-6ed2-45fd-806a-570ac2742a6a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:15.448931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bb98f-6ed2-45fd-806a-570ac2742a6a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:15.449121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bb98f-6ed2-45fd-806a-570ac2742a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:15.527464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-089bb98f-6ed2-45fd-806a-570ac2742a6a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:15.528571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-089bb98f-6ed2-45fd-806a-570ac2742a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:15.640702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 711/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:15.681512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd1f58-2e21-4e80-8441-377f4e76ebfc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:15.683221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-34dd1f58-2e21-4e80-8441-377f4e76ebfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:15.785427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:15.853473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd1f58-2e21-4e80-8441-377f4e76ebfc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:15.980934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f746c69-feb8-40d4-a27e-feaaf442b395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:15.981692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746c69-feb8-40d4-a27e-feaaf442b395 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:15.982231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746c69-feb8-40d4-a27e-feaaf442b395 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:15.982493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746c69-feb8-40d4-a27e-feaaf442b395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:15.984127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1f746c69-feb8-40d4-a27e-feaaf442b395 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:16.048271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:16.088067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f746c69-feb8-40d4-a27e-feaaf442b395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:16.090536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f746c69-feb8-40d4-a27e-feaaf442b395 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:16.112341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 698/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:25.632784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=78 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:26.109951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=22 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:33.249708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b50eac-a01a-4d8f-8069-d738ee2cc42a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.250283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b50eac-a01a-4d8f-8069-d738ee2cc42a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.250539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b50eac-a01a-4d8f-8069-d738ee2cc42a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.253052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 712/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Aug 10 13:31:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:33.258781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1a6e-7d27-4818-97de-365817275083 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.259215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1a6e-7d27-4818-97de-365817275083 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.259329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1a6e-7d27-4818-97de-365817275083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.334775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5e3f1a6e-7d27-4818-97de-365817275083 None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:33.336063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5e3f1a6e-7d27-4818-97de-365817275083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:33.439344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 699/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Aug 10 13:31:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:33.454441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65820439-e52b-4caa-9189-a51bfdb8c55f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.454717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65820439-e52b-4caa-9189-a51bfdb8c55f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.454982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65820439-e52b-4caa-9189-a51bfdb8c55f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.456332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 713/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Aug 10 13:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:33.460826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f998d1b-b292-4883-82c6-14e7588c66bf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:33.461104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f998d1b-b292-4883-82c6-14e7588c66bf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:33.461400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f998d1b-b292-4883-82c6-14e7588c66bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:33.532705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:33.535431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2f998d1b-b292-4883-82c6-14e7588c66bf None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:33.537207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2f998d1b-b292-4883-82c6-14e7588c66bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:33.651474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 700/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Aug 10 13:31:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:36.117539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1e2c-5fac-4b3d-a14a-eeb53b2db13d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:36.118074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1e2c-5fac-4b3d-a14a-eeb53b2db13d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:36.118074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1e2c-5fac-4b3d-a14a-eeb53b2db13d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:36.119338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:36.125085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053b6a0-2cca-4c71-94ca-469a7432f69d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:36.125539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053b6a0-2cca-4c71-94ca-469a7432f69d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:36.125653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053b6a0-2cca-4c71-94ca-469a7432f69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:36.190411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d053b6a0-2cca-4c71-94ca-469a7432f69d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:36.191628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d053b6a0-2cca-4c71-94ca-469a7432f69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:36.245988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:36.286979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:36.330015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e1e2c-5fac-4b3d-a14a-eeb53b2db13d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:36.331521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ff6e1e2c-5fac-4b3d-a14a-eeb53b2db13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:36.473959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e1e2c-5fac-4b3d-a14a-eeb53b2db13d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:36.545501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:36.585333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e82df9-edf4-4457-a3bc-bc4c64f1cb7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:36.585947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e82df9-edf4-4457-a3bc-bc4c64f1cb7b service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:36.586226 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e82df9-edf4-4457-a3bc-bc4c64f1cb7b service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:36.586495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e82df9-edf4-4457-a3bc-bc4c64f1cb7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:36.588407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-31e82df9-edf4-4457-a3bc-bc4c64f1cb7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:36.677564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e82df9-edf4-4457-a3bc-bc4c64f1cb7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:36.679295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e82df9-edf4-4457-a3bc-bc4c64f1cb7b service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:36.696811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 715/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:37.600921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98742516-02ff-4924-93da-e271745f8cfc None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:37.601494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98742516-02ff-4924-93da-e271745f8cfc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:37.601494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98742516-02ff-4924-93da-e271745f8cfc None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:37.602753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:37.608355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4bcde-41bd-4d27-96b3-09b793e05038 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:37.608793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4bcde-41bd-4d27-96b3-09b793e05038 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:37.608901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4bcde-41bd-4d27-96b3-09b793e05038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:37.690891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0ce4bcde-41bd-4d27-96b3-09b793e05038 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:37.692106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0ce4bcde-41bd-4d27-96b3-09b793e05038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:37.717856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:37.786145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:37.936206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98742516-02ff-4924-93da-e271745f8cfc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:37.937784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-98742516-02ff-4924-93da-e271745f8cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:38.081379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98742516-02ff-4924-93da-e271745f8cfc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:38.121811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:38.194062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79f8c0-c4e3-42d8-b97d-95837ad9798c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:38.194590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79f8c0-c4e3-42d8-b97d-95837ad9798c service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:38.194863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79f8c0-c4e3-42d8-b97d-95837ad9798c service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:38.195125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79f8c0-c4e3-42d8-b97d-95837ad9798c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:38.196714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7c79f8c0-c4e3-42d8-b97d-95837ad9798c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:38.284691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79f8c0-c4e3-42d8-b97d-95837ad9798c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:38.286313 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79f8c0-c4e3-42d8-b97d-95837ad9798c service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:38.303764 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 703/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:39.184701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798604f1-9f8a-4a3f-a839-29cd7e0acc8d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:39.185259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798604f1-9f8a-4a3f-a839-29cd7e0acc8d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:39.185259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798604f1-9f8a-4a3f-a839-29cd7e0acc8d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:39.186592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:39.191398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65409d9b-a70f-4c4e-b247-737eadadcd7a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:39.191850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65409d9b-a70f-4c4e-b247-737eadadcd7a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:39.191965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65409d9b-a70f-4c4e-b247-737eadadcd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:39.248254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:39.270329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-65409d9b-a70f-4c4e-b247-737eadadcd7a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:39.271574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-65409d9b-a70f-4c4e-b247-737eadadcd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:39.366006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:39.522765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798604f1-9f8a-4a3f-a839-29cd7e0acc8d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:39.524120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-798604f1-9f8a-4a3f-a839-29cd7e0acc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:39.666798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798604f1-9f8a-4a3f-a839-29cd7e0acc8d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:39.679185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:39.797765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866de09-637f-4d65-801a-abb8ec964b33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:39.798269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866de09-637f-4d65-801a-abb8ec964b33 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:39.798543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866de09-637f-4d65-801a-abb8ec964b33 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:39.798810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866de09-637f-4d65-801a-abb8ec964b33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:39.800390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0866de09-637f-4d65-801a-abb8ec964b33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:39.888280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866de09-637f-4d65-801a-abb8ec964b33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:39.889942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866de09-637f-4d65-801a-abb8ec964b33 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:39.908017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 718/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:40.765455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888b548-849d-4279-8239-edb752296921 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:40.766038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888b548-849d-4279-8239-edb752296921 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:40.766038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888b548-849d-4279-8239-edb752296921 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:40.767368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:31:40.772274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4936-2fd6-415b-b9f9-11ff92742966 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:40.772662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4936-2fd6-415b-b9f9-11ff92742966 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:40.772813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4936-2fd6-415b-b9f9-11ff92742966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:40.792894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:40.849829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9eee4936-2fd6-415b-b9f9-11ff92742966 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:40.851043 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9eee4936-2fd6-415b-b9f9-11ff92742966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:40.950085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:31:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:41.098282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888b548-849d-4279-8239-edb752296921 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:41.099702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1888b548-849d-4279-8239-edb752296921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:41.207691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:41.243145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888b548-849d-4279-8239-edb752296921 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:41.353321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d0ed0-31bd-4fca-b177-f57865d36fc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:41.353843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0ed0-31bd-4fca-b177-f57865d36fc8 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:41.354121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0ed0-31bd-4fca-b177-f57865d36fc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:41.354384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0ed0-31bd-4fca-b177-f57865d36fc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:41.355996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8e1d0ed0-31bd-4fca-b177-f57865d36fc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:41.430416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:41.444537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d0ed0-31bd-4fca-b177-f57865d36fc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:41.446137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d0ed0-31bd-4fca-b177-f57865d36fc8 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:41.463902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 706/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:42.319730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd24ee-1966-44da-89b7-1d92baa01d89 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.320247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd24ee-1966-44da-89b7-1d92baa01d89 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.320247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd24ee-1966-44da-89b7-1d92baa01d89 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.321649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:42.327177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceec52-f23c-4f41-bf2f-c8470fa19218 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.327628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceec52-f23c-4f41-bf2f-c8470fa19218 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.327728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceec52-f23c-4f41-bf2f-c8470fa19218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.404253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-32ceec52-f23c-4f41-bf2f-c8470fa19218 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:42.405494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-32ceec52-f23c-4f41-bf2f-c8470fa19218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:42.499787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:42.649702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd24ee-1966-44da-89b7-1d92baa01d89 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:42.651212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-82cd24ee-1966-44da-89b7-1d92baa01d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:42.733508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:42.794687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd24ee-1966-44da-89b7-1d92baa01d89 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:42.905636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2b464-1766-40fc-a855-18e384597bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:42.906195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2b464-1766-40fc-a855-18e384597bdd service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:42.906473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2b464-1766-40fc-a855-18e384597bdd service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:42.906738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2b464-1766-40fc-a855-18e384597bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:42.908316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-efe2b464-1766-40fc-a855-18e384597bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:42.955393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:42.997082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2b464-1766-40fc-a855-18e384597bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:42.998720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2b464-1766-40fc-a855-18e384597bdd service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:43.016126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 721/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:43.816554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ecf7b-7a5c-454e-a573-b658369ac50d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:43.817254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ecf7b-7a5c-454e-a573-b658369ac50d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:43.817254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ecf7b-7a5c-454e-a573-b658369ac50d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:43.818480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:43.823528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82475d-8929-44a3-b65b-3ac3a8f6dbd4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:43.824033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82475d-8929-44a3-b65b-3ac3a8f6dbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:43.824621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82475d-8929-44a3-b65b-3ac3a8f6dbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:43.899641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ac82475d-8929-44a3-b65b-3ac3a8f6dbd4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:43.900835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ac82475d-8929-44a3-b65b-3ac3a8f6dbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:43.993478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:44.146581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ecf7b-7a5c-454e-a573-b658369ac50d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:44.148103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fe5ecf7b-7a5c-454e-a573-b658369ac50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:44.195825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:44.291509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ecf7b-7a5c-454e-a573-b658369ac50d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:44.402762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6538dbc-e71a-4fbd-88aa-0fe1fcc44086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:44.403276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538dbc-e71a-4fbd-88aa-0fe1fcc44086 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:44.403585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538dbc-e71a-4fbd-88aa-0fe1fcc44086 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:44.403850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538dbc-e71a-4fbd-88aa-0fe1fcc44086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:44.406041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b6538dbc-e71a-4fbd-88aa-0fe1fcc44086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:44.415252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:44.499836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6538dbc-e71a-4fbd-88aa-0fe1fcc44086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:44.501442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6538dbc-e71a-4fbd-88aa-0fe1fcc44086 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:44.519114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 709/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:45.320245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86fa3e-8da1-4769-97c5-9f5727abaabb None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:45.320762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86fa3e-8da1-4769-97c5-9f5727abaabb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:45.320864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86fa3e-8da1-4769-97c5-9f5727abaabb None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:45.322174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:45.327731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947089b8-229f-4c5c-818e-fb5edb68a3a1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:45.328140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947089b8-229f-4c5c-818e-fb5edb68a3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:45.328266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947089b8-229f-4c5c-818e-fb5edb68a3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:45.404097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-947089b8-229f-4c5c-818e-fb5edb68a3a1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:45.405340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-947089b8-229f-4c5c-818e-fb5edb68a3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:45.472425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:45.501914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:45.652365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86fa3e-8da1-4769-97c5-9f5727abaabb None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:45.653904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bb86fa3e-8da1-4769-97c5-9f5727abaabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:45.671094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:45.797860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86fa3e-8da1-4769-97c5-9f5727abaabb None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:45.887212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:45.913194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a6666-410b-4e7d-aac9-ae1d90bd89c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:45.913754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a6666-410b-4e7d-aac9-ae1d90bd89c4 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:45.914027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a6666-410b-4e7d-aac9-ae1d90bd89c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:45.914292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a6666-410b-4e7d-aac9-ae1d90bd89c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:45.915875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fb3a6666-410b-4e7d-aac9-ae1d90bd89c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:46.003470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a6666-410b-4e7d-aac9-ae1d90bd89c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:46.005075 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a6666-410b-4e7d-aac9-ae1d90bd89c4 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:46.024500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 724/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:46.826781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a63c5-0403-48cd-97fd-eb12a0293538 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:46.827366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a63c5-0403-48cd-97fd-eb12a0293538 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:46.827366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a63c5-0403-48cd-97fd-eb12a0293538 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:46.828646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:46.833472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1748a90-8c0c-41fb-995e-8b302caa8108 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:46.833971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1748a90-8c0c-41fb-995e-8b302caa8108 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:46.834067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1748a90-8c0c-41fb-995e-8b302caa8108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:46.909842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c1748a90-8c0c-41fb-995e-8b302caa8108 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:46.911026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c1748a90-8c0c-41fb-995e-8b302caa8108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:47.078136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:47.132346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:47.284768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a63c5-0403-48cd-97fd-eb12a0293538 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:47.286346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a75a63c5-0403-48cd-97fd-eb12a0293538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:47.428630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a63c5-0403-48cd-97fd-eb12a0293538 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:47.481689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:47.539501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336055d-fe76-44cf-8e59-315caacb6e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:47.540121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336055d-fe76-44cf-8e59-315caacb6e44 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:47.540405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336055d-fe76-44cf-8e59-315caacb6e44 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:47.540681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336055d-fe76-44cf-8e59-315caacb6e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:47.542326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6336055d-fe76-44cf-8e59-315caacb6e44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:47.631122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336055d-fe76-44cf-8e59-315caacb6e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:47.632701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336055d-fe76-44cf-8e59-315caacb6e44 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:47.650751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 712/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:48.458026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa69e20-343e-4870-8ddb-633e75da0d0f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:48.458568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa69e20-343e-4870-8ddb-633e75da0d0f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:48.458568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa69e20-343e-4870-8ddb-633e75da0d0f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:48.460121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:48.465083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af118e-5b5a-4ae9-b097-71e433d40ca2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:48.465544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af118e-5b5a-4ae9-b097-71e433d40ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:48.465664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af118e-5b5a-4ae9-b097-71e433d40ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:48.540147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-71af118e-5b5a-4ae9-b097-71e433d40ca2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:48.541292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-71af118e-5b5a-4ae9-b097-71e433d40ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:48.543887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:48.637254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:48.787303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa69e20-343e-4870-8ddb-633e75da0d0f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:48.788814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7fa69e20-343e-4870-8ddb-633e75da0d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:48.930708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa69e20-343e-4870-8ddb-633e75da0d0f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:48.956933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:49.042085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b544c-3376-4bb8-9235-ec7b76116972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:49.042682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b544c-3376-4bb8-9235-ec7b76116972 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:49.042957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b544c-3376-4bb8-9235-ec7b76116972 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:49.043223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b544c-3376-4bb8-9235-ec7b76116972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:49.044792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-369b544c-3376-4bb8-9235-ec7b76116972 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:49.131423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b544c-3376-4bb8-9235-ec7b76116972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:49.133069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b544c-3376-4bb8-9235-ec7b76116972 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:49.150976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 727/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:51.069315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34551ffe-e202-43c7-9028-bf52702097fb None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:51.069817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34551ffe-e202-43c7-9028-bf52702097fb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:51.069934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34551ffe-e202-43c7-9028-bf52702097fb None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:51.071200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:51.076392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e53f-9cd0-430c-94af-b5914889d9d1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:51.076832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e53f-9cd0-430c-94af-b5914889d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:51.076937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e53f-9cd0-430c-94af-b5914889d9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:51.120174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:51.145176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7ff6e53f-9cd0-430c-94af-b5914889d9d1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:51.146394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7ff6e53f-9cd0-430c-94af-b5914889d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:51.240907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:51.280412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34551ffe-e202-43c7-9028-bf52702097fb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:51.282009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-34551ffe-e202-43c7-9028-bf52702097fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:51.383140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34551ffe-e202-43c7-9028-bf52702097fb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:51.415281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:51.455802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b8051-c65b-4951-958b-1b126ed7c53b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:51.456403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b8051-c65b-4951-958b-1b126ed7c53b service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:51.456675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b8051-c65b-4951-958b-1b126ed7c53b service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:51.456940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b8051-c65b-4951-958b-1b126ed7c53b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:51.458557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e05b8051-c65b-4951-958b-1b126ed7c53b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:51.563585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b8051-c65b-4951-958b-1b126ed7c53b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:51.564997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b8051-c65b-4951-958b-1b126ed7c53b service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:51.584691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 715/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:53.725968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cf14f-bca1-45d4-a2a2-aacac07de5c3 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:53.726544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cf14f-bca1-45d4-a2a2-aacac07de5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:53.726544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cf14f-bca1-45d4-a2a2-aacac07de5c3 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:53.727881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:53.733838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d5fb8-0cfc-43dd-8a07-518a780e4b64 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:53.734290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d5fb8-0cfc-43dd-8a07-518a780e4b64 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:53.734397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d5fb8-0cfc-43dd-8a07-518a780e4b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:53.804444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ec3d5fb8-0cfc-43dd-8a07-518a780e4b64 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:53.805997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ec3d5fb8-0cfc-43dd-8a07-518a780e4b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:53.821603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:53.905593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:53.944111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cf14f-bca1-45d4-a2a2-aacac07de5c3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:53.945684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b69cf14f-bca1-45d4-a2a2-aacac07de5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:54.045861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cf14f-bca1-45d4-a2a2-aacac07de5c3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:54.057787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:54.118592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6dc947-d3d7-4ccc-b364-484b5cc8bf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:54.119181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dc947-d3d7-4ccc-b364-484b5cc8bf86 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:54.119452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dc947-d3d7-4ccc-b364-484b5cc8bf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:54.119714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dc947-d3d7-4ccc-b364-484b5cc8bf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:54.121516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-da6dc947-d3d7-4ccc-b364-484b5cc8bf86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:54.227341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6dc947-d3d7-4ccc-b364-484b5cc8bf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:54.228759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6dc947-d3d7-4ccc-b364-484b5cc8bf86 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:54.246666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 730/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:56.634215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998575c8-a4f7-428f-a0ae-12c79094a395 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:56.634751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998575c8-a4f7-428f-a0ae-12c79094a395 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:56.634751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998575c8-a4f7-428f-a0ae-12c79094a395 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:56.636101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:56.641473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f85040-9e2d-4ab6-a10c-017932bc5440 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:56.641887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f85040-9e2d-4ab6-a10c-017932bc5440 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:56.641992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f85040-9e2d-4ab6-a10c-017932bc5440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:56.688692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:56.706088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d3f85040-9e2d-4ab6-a10c-017932bc5440 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:56.707271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d3f85040-9e2d-4ab6-a10c-017932bc5440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:56.799817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:31:56.838607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998575c8-a4f7-428f-a0ae-12c79094a395 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:56.840624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-998575c8-a4f7-428f-a0ae-12c79094a395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:56.942670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998575c8-a4f7-428f-a0ae-12c79094a395 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:56.982071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:57.015085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c73a9b-18f4-4bfc-bf16-507e8d39d6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:57.015689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c73a9b-18f4-4bfc-bf16-507e8d39d6fa service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:57.015961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c73a9b-18f4-4bfc-bf16-507e8d39d6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:57.016224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c73a9b-18f4-4bfc-bf16-507e8d39d6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:57.017919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b4c73a9b-18f4-4bfc-bf16-507e8d39d6fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:57.133206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c73a9b-18f4-4bfc-bf16-507e8d39d6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:57.134583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c73a9b-18f4-4bfc-bf16-507e8d39d6fa service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:57.153775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 718/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:59.276260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f3f04-60b9-40f9-9f25-d691bc054acf None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:59.276260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f3f04-60b9-40f9-9f25-d691bc054acf None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:59.276736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f3f04-60b9-40f9-9f25-d691bc054acf None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:59.278382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:31:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:31:59.283559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec9cf1-cf41-4693-9c79-af83b5667e53 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:59.284014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec9cf1-cf41-4693-9c79-af83b5667e53 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:59.284118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec9cf1-cf41-4693-9c79-af83b5667e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:59.350630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3dec9cf1-cf41-4693-9c79-af83b5667e53 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:31:59.351843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3dec9cf1-cf41-4693-9c79-af83b5667e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:59.372932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:59.448214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:31:59.488413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f3f04-60b9-40f9-9f25-d691bc054acf None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:59.489959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-560f3f04-60b9-40f9-9f25-d691bc054acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:59.592906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f3f04-60b9-40f9-9f25-d691bc054acf None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:59.621892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:59.675204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a30b1c-d520-4af7-ac95-31296d96ac12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:59.675800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a30b1c-d520-4af7-ac95-31296d96ac12 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:59.676074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a30b1c-d520-4af7-ac95-31296d96ac12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:59.676338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a30b1c-d520-4af7-ac95-31296d96ac12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:59.677954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-28a30b1c-d520-4af7-ac95-31296d96ac12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:59.783026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a30b1c-d520-4af7-ac95-31296d96ac12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:59.784443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a30b1c-d520-4af7-ac95-31296d96ac12 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:59.801718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 733/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:01.839353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48ec19-4db3-464a-b3c0-5596776c30d6 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:01.839903 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48ec19-4db3-464a-b3c0-5596776c30d6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:01.840002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48ec19-4db3-464a-b3c0-5596776c30d6 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:01.841370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:01.846494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e5866-fa51-4c7b-829c-f242a8a3b6e0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:01.846880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e5866-fa51-4c7b-829c-f242a8a3b6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:01.847031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e5866-fa51-4c7b-829c-f242a8a3b6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:01.909788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:01.912612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ee0e5866-fa51-4c7b-829c-f242a8a3b6e0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:01.914516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ee0e5866-fa51-4c7b-829c-f242a8a3b6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:02.008291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:02.046658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48ec19-4db3-464a-b3c0-5596776c30d6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:02.048183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-df48ec19-4db3-464a-b3c0-5596776c30d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:02.150480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48ec19-4db3-464a-b3c0-5596776c30d6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:02.198442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:02.227775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99be951-3fcd-4460-951a-ed98a9a42cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:02.228381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99be951-3fcd-4460-951a-ed98a9a42cbc service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:02.228660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99be951-3fcd-4460-951a-ed98a9a42cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:02.228924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99be951-3fcd-4460-951a-ed98a9a42cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:02.230548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b99be951-3fcd-4460-951a-ed98a9a42cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:02.338528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99be951-3fcd-4460-951a-ed98a9a42cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:02.339858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99be951-3fcd-4460-951a-ed98a9a42cbc service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:02.358373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 721/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:04.443469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a632b-6cd5-4e3d-9313-28fbc45606ad None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:04.444028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a632b-6cd5-4e3d-9313-28fbc45606ad None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:04.444028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a632b-6cd5-4e3d-9313-28fbc45606ad None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:04.445389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:04.450773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592b2e-8a73-4f18-9bce-5454a1c4be37 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:04.451190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592b2e-8a73-4f18-9bce-5454a1c4be37 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:04.451309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592b2e-8a73-4f18-9bce-5454a1c4be37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:04.518712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a9592b2e-8a73-4f18-9bce-5454a1c4be37 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:04.519906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a9592b2e-8a73-4f18-9bce-5454a1c4be37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:04.548686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:04.616026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:04.655077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a632b-6cd5-4e3d-9313-28fbc45606ad None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:04.656602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4c6a632b-6cd5-4e3d-9313-28fbc45606ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:04.758682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a632b-6cd5-4e3d-9313-28fbc45606ad None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:04.787852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:04.832388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb37a1b-13b1-481a-82aa-bfaa898d4cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:04.832907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37a1b-13b1-481a-82aa-bfaa898d4cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:04.833178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37a1b-13b1-481a-82aa-bfaa898d4cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:04.833486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37a1b-13b1-481a-82aa-bfaa898d4cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:04.835048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9cb37a1b-13b1-481a-82aa-bfaa898d4cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:04.940773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb37a1b-13b1-481a-82aa-bfaa898d4cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:04.942281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb37a1b-13b1-481a-82aa-bfaa898d4cc1 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:04.961608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 736/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:07.031476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80607a-05d6-4852-98d9-0432ee952f6b None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.032005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80607a-05d6-4852-98d9-0432ee952f6b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.032005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80607a-05d6-4852-98d9-0432ee952f6b None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.033529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:07.038957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc714d-e954-4878-bd41-cee39ec78799 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.039526 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc714d-e954-4878-bd41-cee39ec78799 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.039799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc714d-e954-4878-bd41-cee39ec78799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.110965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-49bc714d-e954-4878-bd41-cee39ec78799 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:07.112178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-49bc714d-e954-4878-bd41-cee39ec78799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:07.121600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:07.212852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:07.251654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80607a-05d6-4852-98d9-0432ee952f6b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:07.253278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fb80607a-05d6-4852-98d9-0432ee952f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:07.366189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80607a-05d6-4852-98d9-0432ee952f6b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:07.423202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:07.444051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc1db8-8a37-410c-90ea-a91e4b435d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:07.444656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1db8-8a37-410c-90ea-a91e4b435d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:07.444947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1db8-8a37-410c-90ea-a91e4b435d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:07.445200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1db8-8a37-410c-90ea-a91e4b435d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:07.446923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-63fc1db8-8a37-410c-90ea-a91e4b435d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:07.559512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc1db8-8a37-410c-90ea-a91e4b435d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:07.560851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc1db8-8a37-410c-90ea-a91e4b435d0a service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:07.580089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 724/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:09.701061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476dec6-81ff-42f4-b820-1aa302e08117 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:09.701660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476dec6-81ff-42f4-b820-1aa302e08117 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:09.702067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476dec6-81ff-42f4-b820-1aa302e08117 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:09.704382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:32:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:09.709763 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4d5b-2ed1-446b-8b6c-c2eb373d48a4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:09.710267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4d5b-2ed1-446b-8b6c-c2eb373d48a4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:09.710364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4d5b-2ed1-446b-8b6c-c2eb373d48a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:09.784073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-63cb4d5b-2ed1-446b-8b6c-c2eb373d48a4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:09.785470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-63cb4d5b-2ed1-446b-8b6c-c2eb373d48a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:09.833757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:09.905296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:09.942532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7476dec6-81ff-42f4-b820-1aa302e08117 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:09.944055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7476dec6-81ff-42f4-b820-1aa302e08117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:10.061301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7476dec6-81ff-42f4-b820-1aa302e08117 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:10.112994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:10.150010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3333df-f424-4ba1-99e6-85bb0b0857c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:10.150621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3333df-f424-4ba1-99e6-85bb0b0857c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:10.150895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3333df-f424-4ba1-99e6-85bb0b0857c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:10.151223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3333df-f424-4ba1-99e6-85bb0b0857c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:10.153458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5b3333df-f424-4ba1-99e6-85bb0b0857c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:10.266925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3333df-f424-4ba1-99e6-85bb0b0857c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:10.268381 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3333df-f424-4ba1-99e6-85bb0b0857c8 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:10.287094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 739/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:12.426809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fedcba-a2b6-4b5a-80d6-c6c21c4c7d5f None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:12.427336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fedcba-a2b6-4b5a-80d6-c6c21c4c7d5f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:12.427612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fedcba-a2b6-4b5a-80d6-c6c21c4c7d5f None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:12.429688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:32:12.435127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c8d6b-1924-4414-bbae-bcf28ccc7c2f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:12.435127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c8d6b-1924-4414-bbae-bcf28ccc7c2f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:12.435652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c8d6b-1924-4414-bbae-bcf28ccc7c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:12.510865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-dd4c8d6b-1924-4414-bbae-bcf28ccc7c2f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:12.512657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dd4c8d6b-1924-4414-bbae-bcf28ccc7c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:12.532216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:12.628660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:12.665397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fedcba-a2b6-4b5a-80d6-c6c21c4c7d5f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:12.666889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f1fedcba-a2b6-4b5a-80d6-c6c21c4c7d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:12.780605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fedcba-a2b6-4b5a-80d6-c6c21c4c7d5f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:12.851588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:12.859121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce54de6-1f9f-467a-add0-85867fc3a6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:12.859729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54de6-1f9f-467a-add0-85867fc3a6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:12.860001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54de6-1f9f-467a-add0-85867fc3a6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:12.860263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54de6-1f9f-467a-add0-85867fc3a6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:12.861863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dce54de6-1f9f-467a-add0-85867fc3a6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:12.974023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce54de6-1f9f-467a-add0-85867fc3a6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:12.975451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce54de6-1f9f-467a-add0-85867fc3a6b3 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:12.996292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 727/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:15.079902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e4fb-2aed-4e98-8b88-a5ae6768b614 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:15.080631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e4fb-2aed-4e98-8b88-a5ae6768b614 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:15.080631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e4fb-2aed-4e98-8b88-a5ae6768b614 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:15.081817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:15.087392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75e64-5c4c-4d6d-af94-5723686e8464 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:15.087993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75e64-5c4c-4d6d-af94-5723686e8464 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:15.088103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75e64-5c4c-4d6d-af94-5723686e8464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:15.158177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-beb75e64-5c4c-4d6d-af94-5723686e8464 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:15.159309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-beb75e64-5c4c-4d6d-af94-5723686e8464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:15.206742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:15.256163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:15.294147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3e4fb-2aed-4e98-8b88-a5ae6768b614 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:15.295692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-63c3e4fb-2aed-4e98-8b88-a5ae6768b614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:15.399423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3e4fb-2aed-4e98-8b88-a5ae6768b614 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:15.446004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:15.472781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc895b-854a-4850-8014-f26acf00d8de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:15.473416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc895b-854a-4850-8014-f26acf00d8de service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:15.473691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc895b-854a-4850-8014-f26acf00d8de service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:15.473957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc895b-854a-4850-8014-f26acf00d8de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:15.475534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-afbc895b-854a-4850-8014-f26acf00d8de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:15.582645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc895b-854a-4850-8014-f26acf00d8de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:15.584084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc895b-854a-4850-8014-f26acf00d8de service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:15.601635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 742/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:25.254448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=22 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:25.599476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=66 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:35.137345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42a61-a8ed-4ed2-b41c-8988c2aca9c2 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:35.137909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42a61-a8ed-4ed2-b41c-8988c2aca9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:35.137909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42a61-a8ed-4ed2-b41c-8988c2aca9c2 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:35.139243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 729/1471] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:35.144395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437df43-4e39-4cc1-bd95-dffae7476fe4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:35.144853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437df43-4e39-4cc1-bd95-dffae7476fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:35.144957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437df43-4e39-4cc1-bd95-dffae7476fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:35.211667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9437df43-4e39-4cc1-bd95-dffae7476fe4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:35.212774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9437df43-4e39-4cc1-bd95-dffae7476fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:35.313070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 743/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:32:35.399473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e9b2-e52f-4bfc-8b42-5cf81b3ce053 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:35.399752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e9b2-e52f-4bfc-8b42-5cf81b3ce053 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:35.400019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e9b2-e52f-4bfc-8b42-5cf81b3ce053 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:35.401384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 730/1473] 10.222.0.52 () {54 vars in 967 bytes} [Wed Aug 10 13:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:35.406853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ab254-3bac-44d8-8c94-cc4022835100 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:35.407129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ab254-3bac-44d8-8c94-cc4022835100 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:35.407395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ab254-3bac-44d8-8c94-cc4022835100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:35.482241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8d4ab254-3bac-44d8-8c94-cc4022835100 None None] MFA Rules not processed for user `ce851cf0509947e783dbd5b4809bc897`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:35.483331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8d4ab254-3bac-44d8-8c94-cc4022835100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:35.503533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:35.559499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 744/1474] 10.222.0.52 () {58 vars in 1064 bytes} [Wed Aug 10 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:35.567008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5e9b2-e52f-4bfc-8b42-5cf81b3ce053 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:35.568510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b0e5e9b2-e52f-4bfc-8b42-5cf81b3ce053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:35.669340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5e9b2-e52f-4bfc-8b42-5cf81b3ce053 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:35.740686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601088ec-d482-4d45-8d93-c17653ecac74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce851cf0509947e783dbd5b4809bc897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:35.741221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601088ec-d482-4d45-8d93-c17653ecac74 service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:35.741522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601088ec-d482-4d45-8d93-c17653ecac74 service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:35.741794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601088ec-d482-4d45-8d93-c17653ecac74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:35.743388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-601088ec-d482-4d45-8d93-c17653ecac74 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:35.797400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:35.851937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601088ec-d482-4d45-8d93-c17653ecac74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:35.853388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601088ec-d482-4d45-8d93-c17653ecac74 service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:35.872602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 731/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:37.962353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601edda-46b5-417b-9a5a-1ddd75674718 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:37.962925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601edda-46b5-417b-9a5a-1ddd75674718 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:37.962925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601edda-46b5-417b-9a5a-1ddd75674718 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:37.964232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 745/1476] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:37.969384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabd45f-189e-4cb9-bae4-12e53aa266f0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:37.969785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabd45f-189e-4cb9-bae4-12e53aa266f0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:37.969930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabd45f-189e-4cb9-bae4-12e53aa266f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:38.035495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5cabd45f-189e-4cb9-bae4-12e53aa266f0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:38.036699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5cabd45f-189e-4cb9-bae4-12e53aa266f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:38.130505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 732/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:38.153715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601edda-46b5-417b-9a5a-1ddd75674718 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:38.155250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9601edda-46b5-417b-9a5a-1ddd75674718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:38.276666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601edda-46b5-417b-9a5a-1ddd75674718 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:38.325357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:38.370969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5d0e8-bfac-419d-8301-c91e1eb5639b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:38.371596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5d0e8-bfac-419d-8301-c91e1eb5639b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:38.371869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5d0e8-bfac-419d-8301-c91e1eb5639b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:38.372132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5d0e8-bfac-419d-8301-c91e1eb5639b admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:38.372757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5d0e8-bfac-419d-8301-c91e1eb5639b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:38.374490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-52e5d0e8-bfac-419d-8301-c91e1eb5639b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:32:38.374490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Aug 10 13:32:38.374490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: Aug 10 13:32:38.375611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5d0e8-bfac-419d-8301-c91e1eb5639b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:38.394192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 746/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 10 13:32:38 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:32:40.431131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283173b-8a91-42f7-b866-27fcdd442fc3 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:40.431665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283173b-8a91-42f7-b866-27fcdd442fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:40.431665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283173b-8a91-42f7-b866-27fcdd442fc3 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:40.433059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 733/1479] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:40.438667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704070-43c4-403c-a136-cab10efa814d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:40.439089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704070-43c4-403c-a136-cab10efa814d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:40.439211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704070-43c4-403c-a136-cab10efa814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:40.505014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-db704070-43c4-403c-a136-cab10efa814d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:40.506258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-db704070-43c4-403c-a136-cab10efa814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:40.601322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 747/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:40.644704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283173b-8a91-42f7-b866-27fcdd442fc3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:40.646255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4283173b-8a91-42f7-b866-27fcdd442fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:40.655930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:40.792641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283173b-8a91-42f7-b866-27fcdd442fc3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:40.870915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:40.903869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458dc12-e7a2-4cba-993c-11c63a9c6c86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:40.904469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458dc12-e7a2-4cba-993c-11c63a9c6c86 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:40.904799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458dc12-e7a2-4cba-993c-11c63a9c6c86 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:40.904993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458dc12-e7a2-4cba-993c-11c63a9c6c86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:40.906608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a458dc12-e7a2-4cba-993c-11c63a9c6c86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:40.992940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458dc12-e7a2-4cba-993c-11c63a9c6c86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:40.994545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458dc12-e7a2-4cba-993c-11c63a9c6c86 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:41.012622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 734/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:43.188366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa5f6-3e34-43a6-8b2c-d99d8d72bf2d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:43.188922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa5f6-3e34-43a6-8b2c-d99d8d72bf2d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:43.188922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa5f6-3e34-43a6-8b2c-d99d8d72bf2d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:43.190201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 748/1482] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:43.195449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871de5bb-34b9-47dc-953a-ef61fa2dc456 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:43.195848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871de5bb-34b9-47dc-953a-ef61fa2dc456 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:43.195985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871de5bb-34b9-47dc-953a-ef61fa2dc456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:43.262381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-871de5bb-34b9-47dc-953a-ef61fa2dc456 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:43.263589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-871de5bb-34b9-47dc-953a-ef61fa2dc456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:43.310363 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:43.361547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 735/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:43.398685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafa5f6-3e34-43a6-8b2c-d99d8d72bf2d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:43.400203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7cafa5f6-3e34-43a6-8b2c-d99d8d72bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:43.423136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:43.502067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafa5f6-3e34-43a6-8b2c-d99d8d72bf2d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:43.573003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6a81-cfc3-45f1-b941-78c8166d5315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:43.573630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6a81-cfc3-45f1-b941-78c8166d5315 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:43.573904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6a81-cfc3-45f1-b941-78c8166d5315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:43.574797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6a81-cfc3-45f1-b941-78c8166d5315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:43.576374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-111b6a81-cfc3-45f1-b941-78c8166d5315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:43.648667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:43.682565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b6a81-cfc3-45f1-b941-78c8166d5315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:43.684698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b6a81-cfc3-45f1-b941-78c8166d5315 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:43.703780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 749/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:45.751401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4854d57-7e86-44e8-8542-42cc7ac7f71e None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:45.752013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4854d57-7e86-44e8-8542-42cc7ac7f71e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:45.752013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4854d57-7e86-44e8-8542-42cc7ac7f71e None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:45.753314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 736/1485] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:45.758619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf63384-07d6-40eb-ab8c-b45d09eb8608 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:45.759042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf63384-07d6-40eb-ab8c-b45d09eb8608 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:45.759143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf63384-07d6-40eb-ab8c-b45d09eb8608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:45.823393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9bf63384-07d6-40eb-ab8c-b45d09eb8608 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:45.824609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9bf63384-07d6-40eb-ab8c-b45d09eb8608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:45.920491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 750/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:45.958912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4854d57-7e86-44e8-8542-42cc7ac7f71e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:45.960438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d4854d57-7e86-44e8-8542-42cc7ac7f71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:46.066041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4854d57-7e86-44e8-8542-42cc7ac7f71e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:46.137159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:46.139284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf66e0a-bbca-424f-9436-743b98d2b347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:46.140115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf66e0a-bbca-424f-9436-743b98d2b347 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:46.140560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf66e0a-bbca-424f-9436-743b98d2b347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:46.141057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf66e0a-bbca-424f-9436-743b98d2b347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:46.142699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8bf66e0a-bbca-424f-9436-743b98d2b347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:46.250997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf66e0a-bbca-424f-9436-743b98d2b347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:46.252554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf66e0a-bbca-424f-9436-743b98d2b347 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:46.271694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 737/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:48.412680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc45a3-8197-4143-8533-614679414adb None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.413264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc45a3-8197-4143-8533-614679414adb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.413264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc45a3-8197-4143-8533-614679414adb None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.414601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 751/1488] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:48.420221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293061fb-8605-4e44-be12-48dec61a14c6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.420661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293061fb-8605-4e44-be12-48dec61a14c6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.420768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293061fb-8605-4e44-be12-48dec61a14c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.486181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-293061fb-8605-4e44-be12-48dec61a14c6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:48.487370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-293061fb-8605-4e44-be12-48dec61a14c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:48.544234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:48.587282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 738/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:48.628049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc45a3-8197-4143-8533-614679414adb None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:48.629662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-68dc45a3-8197-4143-8533-614679414adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:48.665538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:48.732027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc45a3-8197-4143-8533-614679414adb None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:48.803888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a6656-f075-45a0-8ad1-c7130c07e570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:48.804420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a6656-f075-45a0-8ad1-c7130c07e570 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:48.804697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a6656-f075-45a0-8ad1-c7130c07e570 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:48.804966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a6656-f075-45a0-8ad1-c7130c07e570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:48.806578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c07a6656-f075-45a0-8ad1-c7130c07e570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:48.887309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:48.917008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a6656-f075-45a0-8ad1-c7130c07e570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:48.918515 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a6656-f075-45a0-8ad1-c7130c07e570 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:48.936115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 752/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:32:51.159356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162875e8-ec23-4539-a194-13485dc7923c None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:51.159874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162875e8-ec23-4539-a194-13485dc7923c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:51.159874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162875e8-ec23-4539-a194-13485dc7923c None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:51.161159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 739/1491] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 10 13:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:32:51.166371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b227585-aa90-4c3d-bf1e-28757c6cda20 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:51.166827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b227585-aa90-4c3d-bf1e-28757c6cda20 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:51.166933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b227585-aa90-4c3d-bf1e-28757c6cda20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:51.233160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5b227585-aa90-4c3d-bf1e-28757c6cda20 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:32:51.234637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5b227585-aa90-4c3d-bf1e-28757c6cda20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:51.330366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 753/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 10 13:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:32:51.370162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162875e8-ec23-4539-a194-13485dc7923c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:32:51.371701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-162875e8-ec23-4539-a194-13485dc7923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:51.472992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162875e8-ec23-4539-a194-13485dc7923c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:32:51.544450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828834c-532a-4f22-8bdb-8bb52ddfad4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:32:51.545042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828834c-532a-4f22-8bdb-8bb52ddfad4f service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:32:51.545339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828834c-532a-4f22-8bdb-8bb52ddfad4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:32:51.545609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828834c-532a-4f22-8bdb-8bb52ddfad4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:32:51.547166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6828834c-532a-4f22-8bdb-8bb52ddfad4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:32:51.555947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:32:51.673865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828834c-532a-4f22-8bdb-8bb52ddfad4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:32:51.675283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828834c-532a-4f22-8bdb-8bb52ddfad4f service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:32:51.693457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 740/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:33:01.328542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=82 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:33:01.691054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=50 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:02.772214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27872f96-9830-4625-968f-a70a2e011dec None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:02.772828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27872f96-9830-4625-968f-a70a2e011dec None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:02.772828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27872f96-9830-4625-968f-a70a2e011dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:02.847028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-27872f96-9830-4625-968f-a70a2e011dec None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:02.848216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-27872f96-9830-4625-968f-a70a2e011dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:02.942273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 754/1494] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:02.950420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224754-8e5a-44d5-8da6-22070943fc2c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:02.950935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224754-8e5a-44d5-8da6-22070943fc2c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:02.950935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224754-8e5a-44d5-8da6-22070943fc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:03.025441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-48224754-8e5a-44d5-8da6-22070943fc2c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:03.027138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-48224754-8e5a-44d5-8da6-22070943fc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:03.121655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 741/1495] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:03.126293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27872f96-9830-4625-968f-a70a2e011dec None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:03.127521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-27872f96-9830-4625-968f-a70a2e011dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:03.139231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27872f96-9830-4625-968f-a70a2e011dec None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:03.198724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:03.233015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:03.233648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:03.233927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:03.234197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:03.234831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:34:03.235126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:03.236390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:34:03.236390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Aug 10 13:34:03.236390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: Aug 10 13:34:03.237294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:03.248379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 755/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:34:03.256664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48224754-8e5a-44d5-8da6-22070943fc2c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:03.266506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48224754-8e5a-44d5-8da6-22070943fc2c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:03.327037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:03.361203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:03.361841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:03.362123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:03.362393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:03.363038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:03.364751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:34:03.364751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Aug 10 13:34:03.364751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: Aug 10 13:34:03.365982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:03.422293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 742/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:34:03 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:03.426452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:03.427990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:03.461365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321648-4a87-4a05-b5da-606ad57304b9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:03.553667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:03.554195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:03.554469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:03.554736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:03.555352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:03.556356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:34:03.556356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Aug 10 13:34:03.556356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: Aug 10 13:34:03.558102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:03.598653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 756/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:34:03 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:34:03.602682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:03.604169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:03.665011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5a979-7c7c-4362-9a1a-e7335cab8a11 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:03.746639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:03.757393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:03.757998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:03.758276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:03.758536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:03.759157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:03.760149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:34:03.760149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Aug 10 13:34:03.760149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: Aug 10 13:34:03.761607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:03.773855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 743/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:03.777893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:03.779382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:03.790493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57591237-2495-47de-a55e-d634df05c709 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:03.815319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:03.886032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff026e-563b-4745-96ea-f4b79df12576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:03.886631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff026e-563b-4745-96ea-f4b79df12576 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:03.886907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff026e-563b-4745-96ea-f4b79df12576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:03.887181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff026e-563b-4745-96ea-f4b79df12576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:03.887811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff026e-563b-4745-96ea-f4b79df12576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:03.888821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-29ff026e-563b-4745-96ea-f4b79df12576 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:34:03.888821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Aug 10 13:34:03.888821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: Aug 10 13:34:03.889735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff026e-563b-4745-96ea-f4b79df12576 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:03.903207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 757/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:34:03.907415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:03.908950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:03.942608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71566c4-0c8e-433e-8938-4aa296f39703 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:04.033832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96334ec3-263e-4fbd-8d61-cb9fe9caec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:04.034505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334ec3-263e-4fbd-8d61-cb9fe9caec77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:04.034765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334ec3-263e-4fbd-8d61-cb9fe9caec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:04.035030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334ec3-263e-4fbd-8d61-cb9fe9caec77 admin admin] PATH_INFO: `/v3/projects/76597170a61842668a6bb68b19a77519/users/14e1fff9ad2d4db1bc0e8469c713e4e3/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:04.057095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96334ec3-263e-4fbd-8d61-cb9fe9caec77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76597170a61842668a6bb68b19a77519, user_id=14e1fff9ad2d4db1bc0e8469c713e4e3, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:04.058751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-96334ec3-263e-4fbd-8d61-cb9fe9caec77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:34:04.058751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Aug 10 13:34:04.058751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: Aug 10 13:34:04.060725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96334ec3-263e-4fbd-8d61-cb9fe9caec77 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:04.084788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 744/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:34:03 2022] PUT /identity/v3/projects/76597170a61842668a6bb68b19a77519/users/14e1fff9ad2d4db1bc0e8469c713e4e3/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:04.097388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e4d88-018c-4a29-810f-332214e5da71 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:04.097695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e4d88-018c-4a29-810f-332214e5da71 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:04.097962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e4d88-018c-4a29-810f-332214e5da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:04.154535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-de8e4d88-018c-4a29-810f-332214e5da71 None None] MFA Rules not processed for user `14e1fff9ad2d4db1bc0e8469c713e4e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:04.155718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-de8e4d88-018c-4a29-810f-332214e5da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:04.233543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 758/1502] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:34:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:04.249660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccff65e-c1a5-4f4e-ad2c-ad45354cf330 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:04.249952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccff65e-c1a5-4f4e-ad2c-ad45354cf330 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:04.250230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccff65e-c1a5-4f4e-ad2c-ad45354cf330 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:04.251577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 745/1503] 10.222.0.52 () {54 vars in 964 bytes} [Wed Aug 10 13:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:34:04.256639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94384c-2812-4411-af52-2f088c8e9d98 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:04.256946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94384c-2812-4411-af52-2f088c8e9d98 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:04.257239 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94384c-2812-4411-af52-2f088c8e9d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:04.288744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:04.332803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ce94384c-2812-4411-af52-2f088c8e9d98 None None] MFA Rules not processed for user `b96be241231247b39e544529a4cd0b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:04.334064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ce94384c-2812-4411-af52-2f088c8e9d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:04.408792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 759/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Aug 10 13:34:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:34:04.416578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccff65e-c1a5-4f4e-ad2c-ad45354cf330 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:04.418711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eccff65e-c1a5-4f4e-ad2c-ad45354cf330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:04.467719 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:04.521176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccff65e-c1a5-4f4e-ad2c-ad45354cf330 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:04.593395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:04.593995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:04.594266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:04.594527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:04.596082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:04.691243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:04.692582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:04.696013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:04.710801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 746/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:04.755081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4bf1-85fc-4b19-8fc0-7ebaf748cc49 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:04.755384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4bf1-85fc-4b19-8fc0-7ebaf748cc49 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:04.755654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade4bf1-85fc-4b19-8fc0-7ebaf748cc49 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:04.756922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 760/1506] 10.222.0.52 () {52 vars in 830 bytes} [Wed Aug 10 13:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:34:04.760929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:04.762217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:04.858941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9b70f-1312-48ac-bea4-96648322f964 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:04.950577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:04.951131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:04.951406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:04.951669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] PATH_INFO: `/v3/users/14e1fff9ad2d4db1bc0e8469c713e4e3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:04.964658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e1fff9ad2d4db1bc0e8469c713e4e3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:04.965718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:34:04.965718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Aug 10 13:34:04.965718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: Aug 10 13:34:04.966929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:04.976536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:05.031319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e1fff9ad2d4db1bc0e8469c713e4e3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.040434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e1fff9ad2d4db1bc0e8469c713e4e3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.041013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e1fff9ad2d4db1bc0e8469c713e4e3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.054326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e1fff9ad2d4db1bc0e8469c713e4e3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.077456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e1fff9ad2d4db1bc0e8469c713e4e3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:05.082937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 747/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:34:04 2022] DELETE /identity/v3/users/14e1fff9ad2d4db1bc0e8469c713e4e3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:05.087460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db5563-6b7f-4a22-934c-ee24e0d04dd1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:05.087931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db5563-6b7f-4a22-934c-ee24e0d04dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:05.088045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db5563-6b7f-4a22-934c-ee24e0d04dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:05.145944 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e1db5563-6b7f-4a22-934c-ee24e0d04dd1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:34:05.147137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e1db5563-6b7f-4a22-934c-ee24e0d04dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:05.220396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:05.232990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 761/1508] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:34:05.242839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:05.244037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:05.344495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13c6b1-d5ae-4003-8ba5-599995ad21d9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:05.399456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:05.420659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7444a-e3b0-4644-b71f-ab806d6d279a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:05.421291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7444a-e3b0-4644-b71f-ab806d6d279a service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:05.421569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7444a-e3b0-4644-b71f-ab806d6d279a service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:05.421835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7444a-e3b0-4644-b71f-ab806d6d279a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:05.423434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-54e7444a-e3b0-4644-b71f-ab806d6d279a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:05.531067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7444a-e3b0-4644-b71f-ab806d6d279a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:05.532423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7444a-e3b0-4644-b71f-ab806d6d279a service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:05.550998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 748/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:34:05.869563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db5563-6b7f-4a22-934c-ee24e0d04dd1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:34:05.871331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e1db5563-6b7f-4a22-934c-ee24e0d04dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:34:05.882749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db5563-6b7f-4a22-934c-ee24e0d04dd1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:34:05.984903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:34:05.985811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:34:05.986085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:34:05.986356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] PATH_INFO: `/v3/projects/76597170a61842668a6bb68b19a77519` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:34:05.996022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76597170a61842668a6bb68b19a77519)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:34:05.997035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:34:05.997035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Aug 10 13:34:05.997035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: Aug 10 13:34:05.998336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:34:06.059013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76597170a61842668a6bb68b19a77519', 'request_id': 'req-3b3f8d5e-701d-4603-874f-0ca0b61b2089'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.064830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76597170a61842668a6bb68b19a77519', 'request_id': 'req-3b3f8d5e-701d-4603-874f-0ca0b61b2089'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.077654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76597170a61842668a6bb68b19a77519', 'request_id': 'req-3b3f8d5e-701d-4603-874f-0ca0b61b2089'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.102254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] The token cache is being invalidate because project 76597170a61842668a6bb68b19a77519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:34:06.102870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.102990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b3f8d5e-701d-4603-874f-0ca0b61b2089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:34:06.104955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 762/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:34:05 2022] DELETE /identity/v3/projects/76597170a61842668a6bb68b19a77519 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:34:15.548841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=62 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:34:16.100287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:35:33.596214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3e523-65c8-4b6c-840c-2a2114e27daf None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:33.596778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3e523-65c8-4b6c-840c-2a2114e27daf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:33.596778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3e523-65c8-4b6c-840c-2a2114e27daf None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:33.598033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:35:33.602808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e8e0b-9f70-475e-88b5-685b3a334308 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:33.603796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e8e0b-9f70-475e-88b5-685b3a334308 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:33.603939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e8e0b-9f70-475e-88b5-685b3a334308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:33.684165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e91e8e0b-9f70-475e-88b5-685b3a334308 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:33.685402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e91e8e0b-9f70-475e-88b5-685b3a334308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:33.784955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:35:33.937360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3e523-65c8-4b6c-840c-2a2114e27daf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:33.939431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-14a3e523-65c8-4b6c-840c-2a2114e27daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:34.055846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3e523-65c8-4b6c-840c-2a2114e27daf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:34.061825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5648b-b9fa-48ac-ac08-c82f74bc87ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:34.062425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5648b-b9fa-48ac-ac08-c82f74bc87ac service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:34.062701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5648b-b9fa-48ac-ac08-c82f74bc87ac service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:34.062968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5648b-b9fa-48ac-ac08-c82f74bc87ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:34.070744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5648b-b9fa-48ac-ac08-c82f74bc87ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:34.072350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5648b-b9fa-48ac-ac08-c82f74bc87ac service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:34.081870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 750/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:34.921378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd58d16-f248-401e-88b2-12fb15da49de None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:34.922045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd58d16-f248-401e-88b2-12fb15da49de None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:34.922045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd58d16-f248-401e-88b2-12fb15da49de None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:34.923247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:35:34.927928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b24e2-82db-4637-9a68-35344d30faf1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:34.928330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b24e2-82db-4637-9a68-35344d30faf1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:34.928468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b24e2-82db-4637-9a68-35344d30faf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:34.949436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3e9b24e2-82db-4637-9a68-35344d30faf1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:34.950627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3e9b24e2-82db-4637-9a68-35344d30faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:34.958401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:35.112592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd58d16-f248-401e-88b2-12fb15da49de None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:35.126474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd58d16-f248-401e-88b2-12fb15da49de None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:35.132548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd607528-8b21-4b4a-a846-1c59211b7c1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:35.133129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd607528-8b21-4b4a-a846-1c59211b7c1a service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:35.133430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd607528-8b21-4b4a-a846-1c59211b7c1a service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:35.133697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd607528-8b21-4b4a-a846-1c59211b7c1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:35.141580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd607528-8b21-4b4a-a846-1c59211b7c1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:35.143152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd607528-8b21-4b4a-a846-1c59211b7c1a service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:35.152928 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 765/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:35.645388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c3bb-cc73-4be6-8cad-a0d62f8575f7 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:35.645671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c3bb-cc73-4be6-8cad-a0d62f8575f7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:35.645943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c3bb-cc73-4be6-8cad-a0d62f8575f7 None None] PATH_INFO: `/v3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:35.647224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 752/1517] 10.222.0.42 () {54 vars in 925 bytes} [Wed Aug 10 13:35:35 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:35:35.653333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea2ffc-d6ec-4810-adc2-def2e105908a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:35.653615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea2ffc-d6ec-4810-adc2-def2e105908a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:35.653883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea2ffc-d6ec-4810-adc2-def2e105908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:35.710621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-aeea2ffc-d6ec-4810-adc2-def2e105908a None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:35.711796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aeea2ffc-d6ec-4810-adc2-def2e105908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:35.728851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:35:35.781152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 766/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Aug 10 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:39.542598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11418d3-87ff-423e-8135-32c65ef1d3f1 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:39.543037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11418d3-87ff-423e-8135-32c65ef1d3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:39.543131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11418d3-87ff-423e-8135-32c65ef1d3f1 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:39.544415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 753/1519] 10.222.0.52 () {54 vars in 961 bytes} [Wed Aug 10 13:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:35:39.549806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80d8e9-15b3-4d93-acf8-6e0cf379ca90 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:39.550289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80d8e9-15b3-4d93-acf8-6e0cf379ca90 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:39.550437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80d8e9-15b3-4d93-acf8-6e0cf379ca90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:39.569924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1f80d8e9-15b3-4d93-acf8-6e0cf379ca90 None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:39.571112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1f80d8e9-15b3-4d93-acf8-6e0cf379ca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:39.580107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 767/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Aug 10 13:35:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:39.587378 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11418d3-87ff-423e-8135-32c65ef1d3f1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:39.600361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11418d3-87ff-423e-8135-32c65ef1d3f1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:39.605450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a55b9-a0e1-401b-b02b-a42afc318ee0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:39.606036 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a55b9-a0e1-401b-b02b-a42afc318ee0 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:39.606308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a55b9-a0e1-401b-b02b-a42afc318ee0 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:39.606571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a55b9-a0e1-401b-b02b-a42afc318ee0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:39.614911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a55b9-a0e1-401b-b02b-a42afc318ee0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:39.616467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a55b9-a0e1-401b-b02b-a42afc318ee0 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:39.626322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 754/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:46.395581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648f344-e219-4c1f-95df-b896a935d4fe None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:46.396177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648f344-e219-4c1f-95df-b896a935d4fe None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:46.396177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648f344-e219-4c1f-95df-b896a935d4fe None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:46.397571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:35:46.403042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d079-3ec4-4cae-81eb-e4a3949a1a60 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:46.403470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d079-3ec4-4cae-81eb-e4a3949a1a60 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:46.403575 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d079-3ec4-4cae-81eb-e4a3949a1a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:46.422909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-88c4d079-3ec4-4cae-81eb-e4a3949a1a60 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:46.424077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-88c4d079-3ec4-4cae-81eb-e4a3949a1a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:46.431800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:46.589170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648f344-e219-4c1f-95df-b896a935d4fe None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:46.604163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648f344-e219-4c1f-95df-b896a935d4fe None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:46.609596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9b765-33b3-4288-868f-ad2b141a7863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:46.610178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b765-33b3-4288-868f-ad2b141a7863 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:46.610451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b765-33b3-4288-868f-ad2b141a7863 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:46.610718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b765-33b3-4288-868f-ad2b141a7863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:46.618833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9b765-33b3-4288-868f-ad2b141a7863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:46.620408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9b765-33b3-4288-868f-ad2b141a7863 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:46.629618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 769/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:47.447390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35f45-bdde-46f9-9e0d-dba7c4777d2a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:47.447974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35f45-bdde-46f9-9e0d-dba7c4777d2a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:47.447974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35f45-bdde-46f9-9e0d-dba7c4777d2a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:47.449358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 13:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:35:47.454172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08828567-bb52-48a7-926a-af6be8e2f954 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:47.454600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08828567-bb52-48a7-926a-af6be8e2f954 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:47.454711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08828567-bb52-48a7-926a-af6be8e2f954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:47.473391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-08828567-bb52-48a7-926a-af6be8e2f954 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:47.474574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-08828567-bb52-48a7-926a-af6be8e2f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:47.482857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 13:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:47.635977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd35f45-bdde-46f9-9e0d-dba7c4777d2a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:47.650391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd35f45-bdde-46f9-9e0d-dba7c4777d2a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:47.655716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8465733-f931-4982-81f2-a2cc69819416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:47.656335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465733-f931-4982-81f2-a2cc69819416 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:47.656612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465733-f931-4982-81f2-a2cc69819416 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:47.656881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465733-f931-4982-81f2-a2cc69819416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:47.665084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8465733-f931-4982-81f2-a2cc69819416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:47.666913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8465733-f931-4982-81f2-a2cc69819416 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:47.676154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 757/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:51.636624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852870b8-13a5-4e7c-b79b-01b4726a66c0 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:51.637194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852870b8-13a5-4e7c-b79b-01b4726a66c0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:51.637194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852870b8-13a5-4e7c-b79b-01b4726a66c0 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:51.638548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:35:51.643618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945492d-2800-489b-9481-7d1ac4f6429c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:51.644077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945492d-2800-489b-9481-7d1ac4f6429c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:51.644182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945492d-2800-489b-9481-7d1ac4f6429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:51.662862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2945492d-2800-489b-9481-7d1ac4f6429c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:51.664036 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2945492d-2800-489b-9481-7d1ac4f6429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:51.671725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:51.743767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852870b8-13a5-4e7c-b79b-01b4726a66c0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:51.745527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-852870b8-13a5-4e7c-b79b-01b4726a66c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:51.821505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852870b8-13a5-4e7c-b79b-01b4726a66c0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:51.825544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3247f64-2e03-421b-83a0-5cfd40aec246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:51.826275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3247f64-2e03-421b-83a0-5cfd40aec246 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:51.826713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3247f64-2e03-421b-83a0-5cfd40aec246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:51.827095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3247f64-2e03-421b-83a0-5cfd40aec246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:51.836123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3247f64-2e03-421b-83a0-5cfd40aec246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:51.838080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3247f64-2e03-421b-83a0-5cfd40aec246 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:51.847276 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 772/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:53.825746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a756-163f-4049-a9ed-b887090f6e16 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:53.826293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a756-163f-4049-a9ed-b887090f6e16 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:53.826417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a756-163f-4049-a9ed-b887090f6e16 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:53.828011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:35:53.833616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd6d9-0e62-4b77-933e-7ef85154b958 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:53.834034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd6d9-0e62-4b77-933e-7ef85154b958 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:53.834151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd6d9-0e62-4b77-933e-7ef85154b958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:53.853156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7d2cd6d9-0e62-4b77-933e-7ef85154b958 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:53.854349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7d2cd6d9-0e62-4b77-933e-7ef85154b958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:53.862332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:53.897920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746a756-163f-4049-a9ed-b887090f6e16 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:53.908589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746a756-163f-4049-a9ed-b887090f6e16 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:53.912119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfbbb5-0b1f-40e7-94a2-b00c668fc096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:53.912712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbbb5-0b1f-40e7-94a2-b00c668fc096 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:53.912988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbbb5-0b1f-40e7-94a2-b00c668fc096 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:53.913288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbbb5-0b1f-40e7-94a2-b00c668fc096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:53.922045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfbbb5-0b1f-40e7-94a2-b00c668fc096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:53.923461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfbbb5-0b1f-40e7-94a2-b00c668fc096 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:53.933182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 760/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:57.500845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ffdd0-2b3c-47ee-959f-0476f7bfe372 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:57.501430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ffdd0-2b3c-47ee-959f-0476f7bfe372 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:57.501430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ffdd0-2b3c-47ee-959f-0476f7bfe372 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:57.502830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:35:57.507869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faca46-bfe9-4fd4-b501-a357d64fd87e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:57.508296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faca46-bfe9-4fd4-b501-a357d64fd87e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:57.508399 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faca46-bfe9-4fd4-b501-a357d64fd87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:57.516044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=81 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:35:57.530409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-56faca46-bfe9-4fd4-b501-a357d64fd87e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:35:57.531536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-56faca46-bfe9-4fd4-b501-a357d64fd87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:57.539484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:35:57.884712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ffdd0-2b3c-47ee-959f-0476f7bfe372 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:35:57.886405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a22ffdd0-2b3c-47ee-959f-0476f7bfe372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:35:57.949518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:35:57.962889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ffdd0-2b3c-47ee-959f-0476f7bfe372 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:35:57.966360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79f342-4da2-40a1-9c3c-9bbd77d8f568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:35:57.966948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f342-4da2-40a1-9c3c-9bbd77d8f568 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:57.967220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f342-4da2-40a1-9c3c-9bbd77d8f568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:57.967485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79f342-4da2-40a1-9c3c-9bbd77d8f568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:35:57.976029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79f342-4da2-40a1-9c3c-9bbd77d8f568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:35:57.977954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79f342-4da2-40a1-9c3c-9bbd77d8f568 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:35:57.987213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 775/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:35:59.999116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afc6bb-35a9-47cf-b170-df6b29cb7048 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:35:59.999697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afc6bb-35a9-47cf-b170-df6b29cb7048 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:35:59.999697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afc6bb-35a9-47cf-b170-df6b29cb7048 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:00.000987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 13:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:36:00.006045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643fa69-02dd-4589-bd9b-894f494e41d3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:00.006464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643fa69-02dd-4589-bd9b-894f494e41d3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:00.006588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643fa69-02dd-4589-bd9b-894f494e41d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:00.025932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9643fa69-02dd-4589-bd9b-894f494e41d3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:36:00.027116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9643fa69-02dd-4589-bd9b-894f494e41d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:00.034769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 13:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:36:00.364270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afc6bb-35a9-47cf-b170-df6b29cb7048 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:36:00.365986 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f1afc6bb-35a9-47cf-b170-df6b29cb7048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:36:00.379989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afc6bb-35a9-47cf-b170-df6b29cb7048 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:36:00.383500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b44cc5-427f-46f8-93ff-4826a9b63a21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:36:00.384087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44cc5-427f-46f8-93ff-4826a9b63a21 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:36:00.384365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44cc5-427f-46f8-93ff-4826a9b63a21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:36:00.384629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44cc5-427f-46f8-93ff-4826a9b63a21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:36:00.393711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b44cc5-427f-46f8-93ff-4826a9b63a21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:36:00.395151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b44cc5-427f-46f8-93ff-4826a9b63a21 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:36:00.404542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 763/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:36:10.021190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=11 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:36:10.400424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=8 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:24.391705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b6973-f72d-4c31-83d7-1c9193d086f4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.392354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b6973-f72d-4c31-83d7-1c9193d086f4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.392354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b6973-f72d-4c31-83d7-1c9193d086f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.434356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-848b6973-f72d-4c31-83d7-1c9193d086f4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:24.435592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-848b6973-f72d-4c31-83d7-1c9193d086f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:24.446954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 777/1540] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:24.456005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686544f-ee02-426c-bd16-9c24dc8d9709 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.456597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686544f-ee02-426c-bd16-9c24dc8d9709 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.456597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686544f-ee02-426c-bd16-9c24dc8d9709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.484472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a686544f-ee02-426c-bd16-9c24dc8d9709 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:24.485810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a686544f-ee02-426c-bd16-9c24dc8d9709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:24.500503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 764/1541] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:24.504793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b6973-f72d-4c31-83d7-1c9193d086f4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:24.520605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b6973-f72d-4c31-83d7-1c9193d086f4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:24.533322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:24.533866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.534529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.534806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.535462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:24.535756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:24.540655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:24.558797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 778/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:37:24.567100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686544f-ee02-426c-bd16-9c24dc8d9709 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:24.582616 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686544f-ee02-426c-bd16-9c24dc8d9709 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:24.590838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a63e7-e02c-49da-89d0-2841f51eb8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:24.591619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a63e7-e02c-49da-89d0-2841f51eb8ef admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.591940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a63e7-e02c-49da-89d0-2841f51eb8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.592205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a63e7-e02c-49da-89d0-2841f51eb8ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.592838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a63e7-e02c-49da-89d0-2841f51eb8ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:24.595027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a63e7-e02c-49da-89d0-2841f51eb8ef admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:24.657819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 765/1543] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:24 2022] POST /identity/v3/projects => generated 405 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:24.661878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:24.680517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e44d-cc32-447a-92ea-0ec2d703f910 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:24.774963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da12e60-aa8f-4da9-8372-742f8519d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:24.775586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12e60-aa8f-4da9-8372-742f8519d4ee admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.775857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12e60-aa8f-4da9-8372-742f8519d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.776116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12e60-aa8f-4da9-8372-742f8519d4ee admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.776760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da12e60-aa8f-4da9-8372-742f8519d4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:24.778701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da12e60-aa8f-4da9-8372-742f8519d4ee admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:24.822630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 779/1544] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:24 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:24.827487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a63e7-e02c-49da-89d0-2841f51eb8ef admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:24.847107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a63e7-e02c-49da-89d0-2841f51eb8ef admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:24.859029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6604c8-546e-4fcc-be30-c06a0e558183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:24.859838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6604c8-546e-4fcc-be30-c06a0e558183 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.860152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6604c8-546e-4fcc-be30-c06a0e558183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.860491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6604c8-546e-4fcc-be30-c06a0e558183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.861078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6604c8-546e-4fcc-be30-c06a0e558183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:24.863112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6604c8-546e-4fcc-be30-c06a0e558183 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:24.876571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc4e00-2130-42f8-b791-1c4921600df8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.876897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc4e00-2130-42f8-b791-1c4921600df8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.877120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc4e00-2130-42f8-b791-1c4921600df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.877386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 766/1545] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:24.881581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6604c8-546e-4fcc-be30-c06a0e558183 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:24.896109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6604c8-546e-4fcc-be30-c06a0e558183 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:24.910648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a52959-800f-4fd2-b1db-7bdd5f61d328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:24.911275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a52959-800f-4fd2-b1db-7bdd5f61d328 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.911552 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a52959-800f-4fd2-b1db-7bdd5f61d328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.911818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a52959-800f-4fd2-b1db-7bdd5f61d328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.912456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a52959-800f-4fd2-b1db-7bdd5f61d328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:24.913477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-35dc4e00-2130-42f8-b791-1c4921600df8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:24.914256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a52959-800f-4fd2-b1db-7bdd5f61d328 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:24.914685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-35dc4e00-2130-42f8-b791-1c4921600df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:24.929524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 767/1546] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:37:24.934142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a52959-800f-4fd2-b1db-7bdd5f61d328 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:24.936204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 780/1547] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:24.937481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67791-f5f0-4e7d-8eb9-f4da0d12e6fb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.937757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67791-f5f0-4e7d-8eb9-f4da0d12e6fb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.938023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67791-f5f0-4e7d-8eb9-f4da0d12e6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.949758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a52959-800f-4fd2-b1db-7bdd5f61d328 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:24.966079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb251a-f15a-4237-ac11-9666db544b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:24.966773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb251a-f15a-4237-ac11-9666db544b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:24.967051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb251a-f15a-4237-ac11-9666db544b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:24.967323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb251a-f15a-4237-ac11-9666db544b0f admin admin] PATH_INFO: `/v3/projects/79721e96e7374f97bff559a2ce49aacb/users/f87f0cfd08584af49ee60cd7ad115eb4/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:24.982270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a7f67791-f5f0-4e7d-8eb9-f4da0d12e6fb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:24.983486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a7f67791-f5f0-4e7d-8eb9-f4da0d12e6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:24.987306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb251a-f15a-4237-ac11-9666db544b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=79721e96e7374f97bff559a2ce49aacb, user_id=f87f0cfd08584af49ee60cd7ad115eb4, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:24.990068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb251a-f15a-4237-ac11-9666db544b0f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.002774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 781/1548] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.004120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179eee8-92d6-41ff-b8f7-c3cdb219ca6f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.004404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179eee8-92d6-41ff-b8f7-c3cdb219ca6f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.004673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179eee8-92d6-41ff-b8f7-c3cdb219ca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.030705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 768/1549] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:24 2022] PUT /identity/v3/projects/79721e96e7374f97bff559a2ce49aacb/users/f87f0cfd08584af49ee60cd7ad115eb4/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:37:25.032574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5ca75-7d7a-4c74-a9cf-8489b8c656e0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.032861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5ca75-7d7a-4c74-a9cf-8489b8c656e0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.033128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5ca75-7d7a-4c74-a9cf-8489b8c656e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.046334 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8179eee8-92d6-41ff-b8f7-c3cdb219ca6f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.051739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8179eee8-92d6-41ff-b8f7-c3cdb219ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.083662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b2f5ca75-7d7a-4c74-a9cf-8489b8c656e0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.086241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b2f5ca75-7d7a-4c74-a9cf-8489b8c656e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.173926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 769/1550] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.174423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 782/1551] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.175953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddab819-3412-43e8-94ab-68bc372de1ea None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.181075 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddab819-3412-43e8-94ab-68bc372de1ea None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.181589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddab819-3412-43e8-94ab-68bc372de1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.182406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e628-52e3-4fd0-8a92-451cb3f246a7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.182864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e628-52e3-4fd0-8a92-451cb3f246a7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.185633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e628-52e3-4fd0-8a92-451cb3f246a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.206351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fddab819-3412-43e8-94ab-68bc372de1ea None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.208028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2138e628-52e3-4fd0-8a92-451cb3f246a7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.208269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fddab819-3412-43e8-94ab-68bc372de1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.209504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2138e628-52e3-4fd0-8a92-451cb3f246a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.217999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 783/1552] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.222409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 770/1553] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.222409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138e628-52e3-4fd0-8a92-451cb3f246a7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.230759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845dd0e-fd6d-4fee-8736-3528684b35a0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.231320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845dd0e-fd6d-4fee-8736-3528684b35a0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.231770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845dd0e-fd6d-4fee-8736-3528684b35a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.241273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138e628-52e3-4fd0-8a92-451cb3f246a7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.252610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.253255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.253540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.253838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.254485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:25.254785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.255823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:37:25.255823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Aug 10 13:37:25.255823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: Aug 10 13:37:25.256713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.263057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d845dd0e-fd6d-4fee-8736-3528684b35a0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.264745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d845dd0e-fd6d-4fee-8736-3528684b35a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.268570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 771/1554] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:25.269403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.279333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 784/1555] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.280011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb7d78-507b-4b73-9d23-fa7d93980735 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.280947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800cadf-3d00-411d-8957-bf82b6f601bd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.281354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800cadf-3d00-411d-8957-bf82b6f601bd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.281695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800cadf-3d00-411d-8957-bf82b6f601bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.284973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.285758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.285970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.286288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.287161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:25.287466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.289040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.302100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 772/1556] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:25.302272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7800cadf-3d00-411d-8957-bf82b6f601bd None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.303223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.303454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7800cadf-3d00-411d-8957-bf82b6f601bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.304865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.316119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 785/1557] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.317706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a092b-f994-4961-9711-c393bc24a4e3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.317985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a092b-f994-4961-9711-c393bc24a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.318261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a092b-f994-4961-9711-c393bc24a4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.338454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a32a092b-f994-4961-9711-c393bc24a4e3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.339555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a32a092b-f994-4961-9711-c393bc24a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.347203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 786/1558] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.348350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd853f6-1961-46b3-a24c-a3a18047f7d2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.348626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd853f6-1961-46b3-a24c-a3a18047f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.348894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd853f6-1961-46b3-a24c-a3a18047f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.351925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:25.367339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411a383-5f89-468c-b61d-61b81e3963a3 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.368146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5bd853f6-1961-46b3-a24c-a3a18047f7d2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.369344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5bd853f6-1961-46b3-a24c-a3a18047f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.371161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0329f-4b66-4365-bba2-99382e15d957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.371828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0329f-4b66-4365-bba2-99382e15d957 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.372100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0329f-4b66-4365-bba2-99382e15d957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.372362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0329f-4b66-4365-bba2-99382e15d957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.376939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 787/1559] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.378131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a6f2-6b47-4c4b-b1d5-3142c2c187f7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.378518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a6f2-6b47-4c4b-b1d5-3142c2c187f7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.378667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a6f2-6b47-4c4b-b1d5-3142c2c187f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.382015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0329f-4b66-4365-bba2-99382e15d957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.383682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0329f-4b66-4365-bba2-99382e15d957 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.396510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 773/1560] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:25.397514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0329f-4b66-4365-bba2-99382e15d957 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.401457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4d04a6f2-6b47-4c4b-b1d5-3142c2c187f7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.403170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4d04a6f2-6b47-4c4b-b1d5-3142c2c187f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.407714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0329f-4b66-4365-bba2-99382e15d957 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.412235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 788/1561] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.412438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90a010-4024-4a43-a187-c21f0b6fb230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.412988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a010-4024-4a43-a187-c21f0b6fb230 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.413286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a010-4024-4a43-a187-c21f0b6fb230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.413392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900c0d7-50d3-4cd3-a86f-4f3f5be16f81 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.413565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a010-4024-4a43-a187-c21f0b6fb230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.413662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900c0d7-50d3-4cd3-a86f-4f3f5be16f81 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.413916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900c0d7-50d3-4cd3-a86f-4f3f5be16f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.414163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90a010-4024-4a43-a187-c21f0b6fb230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.416617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90a010-4024-4a43-a187-c21f0b6fb230 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.435809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e900c0d7-50d3-4cd3-a86f-4f3f5be16f81 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.437086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e900c0d7-50d3-4cd3-a86f-4f3f5be16f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.446058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 789/1562] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.446717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900c0d7-50d3-4cd3-a86f-4f3f5be16f81 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.456485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900c0d7-50d3-4cd3-a86f-4f3f5be16f81 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.458209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 774/1563] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:25.461503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb9f3-47c2-4d35-b511-a8609c51c0cb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.462783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb9f3-47c2-4d35-b511-a8609c51c0cb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.463088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb9f3-47c2-4d35-b511-a8609c51c0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.467174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:25.489870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b2bcb9f3-47c2-4d35-b511-a8609c51c0cb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.491186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b2bcb9f3-47c2-4d35-b511-a8609c51c0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.529323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.530213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.531218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.531218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.531479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.532521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:37:25.532521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Aug 10 13:37:25.532521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: Aug 10 13:37:25.534020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.564923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 790/1564] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:25.565855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.569029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 775/1565] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.570962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25febeb4-747e-4a5c-8c81-c66332ed06f7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.572017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25febeb4-747e-4a5c-8c81-c66332ed06f7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.572305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25febeb4-747e-4a5c-8c81-c66332ed06f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.599094 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-25febeb4-747e-4a5c-8c81-c66332ed06f7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.600270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-25febeb4-747e-4a5c-8c81-c66332ed06f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.640103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de92c-5c2d-4fd4-97f5-33bf1a9dc11a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.645661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb914c1d-61cf-4bdb-aa1e-97245507021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.646422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb914c1d-61cf-4bdb-aa1e-97245507021d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.646752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb914c1d-61cf-4bdb-aa1e-97245507021d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.647087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb914c1d-61cf-4bdb-aa1e-97245507021d admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.647738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb914c1d-61cf-4bdb-aa1e-97245507021d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:25.648036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb914c1d-61cf-4bdb-aa1e-97245507021d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.650301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb914c1d-61cf-4bdb-aa1e-97245507021d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.663898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 791/1566] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:25.671487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c25531-fac5-4372-bd79-508a1033fcf1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.671773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c25531-fac5-4372-bd79-508a1033fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.672072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c25531-fac5-4372-bd79-508a1033fcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.672610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 776/1567] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.673745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25febeb4-747e-4a5c-8c81-c66332ed06f7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.686603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25febeb4-747e-4a5c-8c81-c66332ed06f7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.691712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.692336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.692611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.692877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.693531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-70c25531-fac5-4372-bd79-508a1033fcf1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:25.693658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:25.693848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.694796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-70c25531-fac5-4372-bd79-508a1033fcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:25.696044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.708641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 792/1568] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:25.709217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 777/1569] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:25.710204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.710687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c25531-fac5-4372-bd79-508a1033fcf1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.722816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f60af-16e6-4ad2-8479-50ac094338f8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.723005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c25531-fac5-4372-bd79-508a1033fcf1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.727628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.727777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ec69e-9cb8-4e9c-b9a7-acec7a959350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.728232 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.728455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ec69e-9cb8-4e9c-b9a7-acec7a959350 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.728555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.728767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.728880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ec69e-9cb8-4e9c-b9a7-acec7a959350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.729107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ec69e-9cb8-4e9c-b9a7-acec7a959350 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.729537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.729937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ec69e-9cb8-4e9c-b9a7-acec7a959350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.730457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:37:25.730457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Aug 10 13:37:25.730457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: Aug 10 13:37:25.731752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.732338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ec69e-9cb8-4e9c-b9a7-acec7a959350 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.756890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 778/1570] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/users => generated 292 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:25.757880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 793/1571] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:25.758011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.759553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ec69e-9cb8-4e9c-b9a7-acec7a959350 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.768848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d02e8-d85b-4ddd-a470-c73db8988634 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.771593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ec69e-9cb8-4e9c-b9a7-acec7a959350 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.773537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.774217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.775081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.775354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.776011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:25.776372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.777953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1abad4-88d0-4ef2-b7a3-8df978567a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.778099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.778649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1abad4-88d0-4ef2-b7a3-8df978567a06 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.779000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1abad4-88d0-4ef2-b7a3-8df978567a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.779333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1abad4-88d0-4ef2-b7a3-8df978567a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.780112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1abad4-88d0-4ef2-b7a3-8df978567a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.782211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1abad4-88d0-4ef2-b7a3-8df978567a06 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.791503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 779/1572] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:25.792159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.803168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8c128-918b-420b-a9e5-e7cab13aaa20 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.808843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.809417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.809772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.809964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.810607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:25.810913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.812523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.813814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 794/1573] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:25.814880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1abad4-88d0-4ef2-b7a3-8df978567a06 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.827487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 780/1574] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:25.828351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.842116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a0890-c201-4922-a309-9a25601bc1ff admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.850019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:25.892748 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:25.919343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1abad4-88d0-4ef2-b7a3-8df978567a06 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.928780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.929637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.929780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.930186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.933685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:25.934000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.935773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.947042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dbc15-709b-40a5-8253-f2d8fcaf6598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.947425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dbc15-709b-40a5-8253-f2d8fcaf6598 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.947727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dbc15-709b-40a5-8253-f2d8fcaf6598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.948303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dbc15-709b-40a5-8253-f2d8fcaf6598 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.948969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dbc15-709b-40a5-8253-f2d8fcaf6598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.951006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dbc15-709b-40a5-8253-f2d8fcaf6598 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.958692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 795/1575] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:25.958692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.970623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f01108-d0c4-49d8-bddd-1aa93829440e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:25.976365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19b7c7-3293-4ab6-83df-10ed6ae19df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:25.977345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19b7c7-3293-4ab6-83df-10ed6ae19df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:25.977697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19b7c7-3293-4ab6-83df-10ed6ae19df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:25.978029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19b7c7-3293-4ab6-83df-10ed6ae19df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:25.978655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19b7c7-3293-4ab6-83df-10ed6ae19df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:25.980839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19b7c7-3293-4ab6-83df-10ed6ae19df0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:25.990833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 781/1576] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:25.990833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dbc15-709b-40a5-8253-f2d8fcaf6598 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:25.998795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 796/1577] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:25.998795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19b7c7-3293-4ab6-83df-10ed6ae19df0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.009751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19b7c7-3293-4ab6-83df-10ed6ae19df0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.062711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dbc15-709b-40a5-8253-f2d8fcaf6598 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.067634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98eac0-46ea-44e7-960b-0f2271ba7c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.068485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98eac0-46ea-44e7-960b-0f2271ba7c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.068759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98eac0-46ea-44e7-960b-0f2271ba7c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.069024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98eac0-46ea-44e7-960b-0f2271ba7c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.069698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98eac0-46ea-44e7-960b-0f2271ba7c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.070918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171b886-ecb4-4a4d-a0a2-ea85b4c05f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.071865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98eac0-46ea-44e7-960b-0f2271ba7c07 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.072167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b886-ecb4-4a4d-a0a2-ea85b4c05f0d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.072448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b886-ecb4-4a4d-a0a2-ea85b4c05f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.072716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b886-ecb4-4a4d-a0a2-ea85b4c05f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.073401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171b886-ecb4-4a4d-a0a2-ea85b4c05f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.075438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171b886-ecb4-4a4d-a0a2-ea85b4c05f0d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.087712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 782/1578] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.087712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98eac0-46ea-44e7-960b-0f2271ba7c07 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.098637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98eac0-46ea-44e7-960b-0f2271ba7c07 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.109519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264904a4-b5d2-40e7-8f80-26e91467201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.110088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264904a4-b5d2-40e7-8f80-26e91467201b admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.110665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264904a4-b5d2-40e7-8f80-26e91467201b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.110665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264904a4-b5d2-40e7-8f80-26e91467201b admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.111285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264904a4-b5d2-40e7-8f80-26e91467201b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.115228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 797/1579] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:25 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.115228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171b886-ecb4-4a4d-a0a2-ea85b4c05f0d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.115494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264904a4-b5d2-40e7-8f80-26e91467201b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.142535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 783/1580] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/users => generated 294 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.143507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264904a4-b5d2-40e7-8f80-26e91467201b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.155458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264904a4-b5d2-40e7-8f80-26e91467201b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.199149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171b886-ecb4-4a4d-a0a2-ea85b4c05f0d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.204298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4f46e-d771-4071-b773-0a1b7741761e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.205295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4f46e-d771-4071-b773-0a1b7741761e admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.205835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4f46e-d771-4071-b773-0a1b7741761e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.206311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4f46e-d771-4071-b773-0a1b7741761e admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.207309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4f46e-d771-4071-b773-0a1b7741761e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.209625 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4f46e-d771-4071-b773-0a1b7741761e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.222521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff71ec-9c49-46df-b9fc-c872819ef15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.223192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff71ec-9c49-46df-b9fc-c872819ef15d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.223872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff71ec-9c49-46df-b9fc-c872819ef15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.223872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff71ec-9c49-46df-b9fc-c872819ef15d admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.224338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff71ec-9c49-46df-b9fc-c872819ef15d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.226318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff71ec-9c49-46df-b9fc-c872819ef15d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.245728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 798/1581] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.251372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4f46e-d771-4071-b773-0a1b7741761e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.253590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:26.266011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 784/1582] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.266011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff71ec-9c49-46df-b9fc-c872819ef15d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.311728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:26.336547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4f46e-d771-4071-b773-0a1b7741761e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.339952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff71ec-9c49-46df-b9fc-c872819ef15d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.342885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce61da-a44c-44fa-b2b5-fbe08edc6408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.343634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce61da-a44c-44fa-b2b5-fbe08edc6408 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.343945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce61da-a44c-44fa-b2b5-fbe08edc6408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.344216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce61da-a44c-44fa-b2b5-fbe08edc6408 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.344857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce61da-a44c-44fa-b2b5-fbe08edc6408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.345130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49157db1-15fb-4e81-beda-9146a96a3c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.345786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49157db1-15fb-4e81-beda-9146a96a3c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.346069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49157db1-15fb-4e81-beda-9146a96a3c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.346343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49157db1-15fb-4e81-beda-9146a96a3c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.346931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce61da-a44c-44fa-b2b5-fbe08edc6408 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.347030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49157db1-15fb-4e81-beda-9146a96a3c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.348620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49157db1-15fb-4e81-beda-9146a96a3c13 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.364144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 785/1583] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.364770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49157db1-15fb-4e81-beda-9146a96a3c13 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.374902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49157db1-15fb-4e81-beda-9146a96a3c13 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.376200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 799/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.377213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce61da-a44c-44fa-b2b5-fbe08edc6408 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.379615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6cb9e-0ac8-44e2-8d86-35b31547c487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.380226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6cb9e-0ac8-44e2-8d86-35b31547c487 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.380507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6cb9e-0ac8-44e2-8d86-35b31547c487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.380779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6cb9e-0ac8-44e2-8d86-35b31547c487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.381456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6cb9e-0ac8-44e2-8d86-35b31547c487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.383061 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6cb9e-0ac8-44e2-8d86-35b31547c487 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.389368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce61da-a44c-44fa-b2b5-fbe08edc6408 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.396996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 786/1585] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.397161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca3418-b01d-461b-96df-01790bcb080e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.397161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca3418-b01d-461b-96df-01790bcb080e admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.397376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6cb9e-0ac8-44e2-8d86-35b31547c487 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.397452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca3418-b01d-461b-96df-01790bcb080e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.397452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca3418-b01d-461b-96df-01790bcb080e admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.397452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca3418-b01d-461b-96df-01790bcb080e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.398028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca3418-b01d-461b-96df-01790bcb080e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.404938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6cb9e-0ac8-44e2-8d86-35b31547c487 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.409726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e5e0-91fa-4023-a3f6-b092f28a59e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.410316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e5e0-91fa-4023-a3f6-b092f28a59e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.410679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e5e0-91fa-4023-a3f6-b092f28a59e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.410862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e5e0-91fa-4023-a3f6-b092f28a59e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.411529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79e5e0-91fa-4023-a3f6-b092f28a59e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.413068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79e5e0-91fa-4023-a3f6-b092f28a59e2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.420296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 800/1586] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.422168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9c8a-210e-4f2a-9708-beb531e9b8b8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.422452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9c8a-210e-4f2a-9708-beb531e9b8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.422752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9c8a-210e-4f2a-9708-beb531e9b8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.424875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 787/1587] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.425643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e5e0-91fa-4023-a3f6-b092f28a59e2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.435937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e5e0-91fa-4023-a3f6-b092f28a59e2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.440545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4bb94-afa1-4cc7-be64-ff2af77629cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.441129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4bb94-afa1-4cc7-be64-ff2af77629cb admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.441455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4bb94-afa1-4cc7-be64-ff2af77629cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.441727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4bb94-afa1-4cc7-be64-ff2af77629cb admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.442337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4bb94-afa1-4cc7-be64-ff2af77629cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.444244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4bb94-afa1-4cc7-be64-ff2af77629cb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.452687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-dede9c8a-210e-4f2a-9708-beb531e9b8b8 None None] MFA Rules not processed for user `f87f0cfd08584af49ee60cd7ad115eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:26.454782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dede9c8a-210e-4f2a-9708-beb531e9b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:26.467122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 788/1588] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/users => generated 303 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.468487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4bb94-afa1-4cc7-be64-ff2af77629cb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.478734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4bb94-afa1-4cc7-be64-ff2af77629cb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.483512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418d856-d777-48b9-95dd-500cf7c5a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.484097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d856-d777-48b9-95dd-500cf7c5a6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.484379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d856-d777-48b9-95dd-500cf7c5a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.484654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d856-d777-48b9-95dd-500cf7c5a6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.485309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418d856-d777-48b9-95dd-500cf7c5a6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.487470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418d856-d777-48b9-95dd-500cf7c5a6a2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.510379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 789/1589] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.511295 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418d856-d777-48b9-95dd-500cf7c5a6a2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.520537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418d856-d777-48b9-95dd-500cf7c5a6a2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.524541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 801/1590] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:26.525399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede9c8a-210e-4f2a-9708-beb531e9b8b8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.528508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53812c5-b7b8-4d26-9152-675525e5c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.529165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53812c5-b7b8-4d26-9152-675525e5c607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.529467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53812c5-b7b8-4d26-9152-675525e5c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.529743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53812c5-b7b8-4d26-9152-675525e5c607 admin admin] PATH_INFO: `/v3/projects/48e5556cbbfd4b00aa4f6055cd6238b2/users/a3d9e5dc719c430eac6b5c8d8b814566/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.535699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede9c8a-210e-4f2a-9708-beb531e9b8b8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.540699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ae90-7e0a-4803-aef9-e3904b0507ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.541314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ae90-7e0a-4803-aef9-e3904b0507ca admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.541589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ae90-7e0a-4803-aef9-e3904b0507ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.541854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ae90-7e0a-4803-aef9-e3904b0507ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.542485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4ae90-7e0a-4803-aef9-e3904b0507ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.544079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4ae90-7e0a-4803-aef9-e3904b0507ca admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.544977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53812c5-b7b8-4d26-9152-675525e5c607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e5556cbbfd4b00aa4f6055cd6238b2, user_id=a3d9e5dc719c430eac6b5c8d8b814566, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.547668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53812c5-b7b8-4d26-9152-675525e5c607 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.554487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 802/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.555514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ae90-7e0a-4803-aef9-e3904b0507ca admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.563371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 790/1592] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:26 2022] PUT /identity/v3/projects/48e5556cbbfd4b00aa4f6055cd6238b2/users/a3d9e5dc719c430eac6b5c8d8b814566/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:26.564175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53812c5-b7b8-4d26-9152-675525e5c607 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.566047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ae90-7e0a-4803-aef9-e3904b0507ca admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.623752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.624318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.624621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.624889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] PATH_INFO: `/v3/projects/18c99d576d184b04b933db13eabb8c5e/users/258ccb4bb15a4747b4c17d2a26645888/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.627267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53812c5-b7b8-4d26-9152-675525e5c607 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.632011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6392b3-530c-44c7-bd42-0955d520691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.632648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6392b3-530c-44c7-bd42-0955d520691e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.633376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6392b3-530c-44c7-bd42-0955d520691e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.634175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6392b3-530c-44c7-bd42-0955d520691e admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.635063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6392b3-530c-44c7-bd42-0955d520691e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.636822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6392b3-530c-44c7-bd42-0955d520691e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.642900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c99d576d184b04b933db13eabb8c5e, user_id=258ccb4bb15a4747b4c17d2a26645888, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.643991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:37:26.643991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Aug 10 13:37:26.643991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: Aug 10 13:37:26.646573 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.647473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 791/1593] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.652465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6392b3-530c-44c7-bd42-0955d520691e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.661831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 803/1594] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:26 2022] PUT /identity/v3/projects/18c99d576d184b04b933db13eabb8c5e/users/258ccb4bb15a4747b4c17d2a26645888/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:26.662771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.666069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6392b3-530c-44c7-bd42-0955d520691e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.671038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:26.675106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:26.726497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb66b1e-fd24-4e2b-ab36-db60a5157087 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.729275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60f769-0cc8-4dce-bae0-b4e68f8b2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.729885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f769-0cc8-4dce-bae0-b4e68f8b2465 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.730157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f769-0cc8-4dce-bae0-b4e68f8b2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.730417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f769-0cc8-4dce-bae0-b4e68f8b2465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.731045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60f769-0cc8-4dce-bae0-b4e68f8b2465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.731169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2184a-4f18-4741-aa85-6fc89e920b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.731749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2184a-4f18-4741-aa85-6fc89e920b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.731993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2184a-4f18-4741-aa85-6fc89e920b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.732264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2184a-4f18-4741-aa85-6fc89e920b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.732891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2184a-4f18-4741-aa85-6fc89e920b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.733104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60f769-0cc8-4dce-bae0-b4e68f8b2465 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.734500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2184a-4f18-4741-aa85-6fc89e920b62 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.743863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 792/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.744837 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 804/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.744990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60f769-0cc8-4dce-bae0-b4e68f8b2465 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.745624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2184a-4f18-4741-aa85-6fc89e920b62 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.754929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60f769-0cc8-4dce-bae0-b4e68f8b2465 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.756978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2184a-4f18-4741-aa85-6fc89e920b62 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.760172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63860db-eb67-4876-b722-c0bc2e16629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.760756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63860db-eb67-4876-b722-c0bc2e16629e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.761029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63860db-eb67-4876-b722-c0bc2e16629e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.761350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63860db-eb67-4876-b722-c0bc2e16629e admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.761938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e606d28-9fe2-4a79-861d-97106d838fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.762076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63860db-eb67-4876-b722-c0bc2e16629e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.762517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e606d28-9fe2-4a79-861d-97106d838fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.762790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e606d28-9fe2-4a79-861d-97106d838fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.763065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e606d28-9fe2-4a79-861d-97106d838fa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.763625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63860db-eb67-4876-b722-c0bc2e16629e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.763720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e606d28-9fe2-4a79-861d-97106d838fa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.765591 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e606d28-9fe2-4a79-861d-97106d838fa1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.773655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 793/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.774475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63860db-eb67-4876-b722-c0bc2e16629e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.784019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63860db-eb67-4876-b722-c0bc2e16629e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.789626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e229f-b473-4d53-b199-17cf2164706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.790172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e229f-b473-4d53-b199-17cf2164706d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.790358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e229f-b473-4d53-b199-17cf2164706d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.790729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e229f-b473-4d53-b199-17cf2164706d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.791501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e229f-b473-4d53-b199-17cf2164706d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.793069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e229f-b473-4d53-b199-17cf2164706d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.798202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 805/1598] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:26.798202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d8a1f-93c1-4fb3-9dda-b58c05389c67 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.798202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d8a1f-93c1-4fb3-9dda-b58c05389c67 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.798202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d8a1f-93c1-4fb3-9dda-b58c05389c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.807601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 794/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.807601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e229f-b473-4d53-b199-17cf2164706d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.814691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e229f-b473-4d53-b199-17cf2164706d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.816199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6a3d8a1f-93c1-4fb3-9dda-b58c05389c67 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:26.817447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6a3d8a1f-93c1-4fb3-9dda-b58c05389c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:26.871589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491b245-61e2-47a7-b59a-aad44306b5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.872192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b245-61e2-47a7-b59a-aad44306b5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.872469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b245-61e2-47a7-b59a-aad44306b5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.872743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b245-61e2-47a7-b59a-aad44306b5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.873459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491b245-61e2-47a7-b59a-aad44306b5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.875065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491b245-61e2-47a7-b59a-aad44306b5b1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.878538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 806/1600] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:26.879842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc3d57-0ab3-472c-b34d-55973eaf4e61 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.880459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc3d57-0ab3-472c-b34d-55973eaf4e61 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.880459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc3d57-0ab3-472c-b34d-55973eaf4e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.889269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 795/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.890146 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491b245-61e2-47a7-b59a-aad44306b5b1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.899813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491b245-61e2-47a7-b59a-aad44306b5b1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.900938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a2fc3d57-0ab3-472c-b34d-55973eaf4e61 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:26.902144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a2fc3d57-0ab3-472c-b34d-55973eaf4e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:26.904303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63afbf-e32a-4b30-af3e-a0a574224ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.904970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63afbf-e32a-4b30-af3e-a0a574224ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.905256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63afbf-e32a-4b30-af3e-a0a574224ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.905541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63afbf-e32a-4b30-af3e-a0a574224ef2 admin admin] PATH_INFO: `/v3/projects/b6cfb6539b69493ebef06cf7176b1935/users/9cecae4323404bd29fe8c05a704de273/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.909786 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 807/1602] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:26.912923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc3d57-0ab3-472c-b34d-55973eaf4e61 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.919632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc3d57-0ab3-472c-b34d-55973eaf4e61 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:26.924116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc63c-6818-4502-b301-a11a58c02866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:26.924722 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc63c-6818-4502-b301-a11a58c02866 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:26.924998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc63c-6818-4502-b301-a11a58c02866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:26.925294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc63c-6818-4502-b301-a11a58c02866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:26.925921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adc63c-6818-4502-b301-a11a58c02866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.926060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63afbf-e32a-4b30-af3e-a0a574224ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6cfb6539b69493ebef06cf7176b1935, user_id=9cecae4323404bd29fe8c05a704de273, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:26.927563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adc63c-6818-4502-b301-a11a58c02866 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.929018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63afbf-e32a-4b30-af3e-a0a574224ef2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:26.938721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 808/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:26.939503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc63c-6818-4502-b301-a11a58c02866 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.943744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 796/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:26 2022] PUT /identity/v3/projects/b6cfb6539b69493ebef06cf7176b1935/users/9cecae4323404bd29fe8c05a704de273/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:26.944820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63afbf-e32a-4b30-af3e-a0a574224ef2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:26.949366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc63c-6818-4502-b301-a11a58c02866 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.001758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:27.005415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757af415-4ba8-413c-828a-089d52f8442f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.006079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757af415-4ba8-413c-828a-089d52f8442f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.006331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757af415-4ba8-413c-828a-089d52f8442f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.006638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757af415-4ba8-413c-828a-089d52f8442f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.007364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757af415-4ba8-413c-828a-089d52f8442f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.008153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63afbf-e32a-4b30-af3e-a0a574224ef2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.009765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757af415-4ba8-413c-828a-089d52f8442f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.013657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e90200-c5a1-4ddc-ae49-938e9f748874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.014257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e90200-c5a1-4ddc-ae49-938e9f748874 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.014547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e90200-c5a1-4ddc-ae49-938e9f748874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.014808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e90200-c5a1-4ddc-ae49-938e9f748874 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.015419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e90200-c5a1-4ddc-ae49-938e9f748874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.017365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e90200-c5a1-4ddc-ae49-938e9f748874 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.019594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 809/1605] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:27.020550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757af415-4ba8-413c-828a-089d52f8442f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.026572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:27.031273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757af415-4ba8-413c-828a-089d52f8442f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.037369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8d3fb-536c-402b-8403-16bd31a10417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.038205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8d3fb-536c-402b-8403-16bd31a10417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.038479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8d3fb-536c-402b-8403-16bd31a10417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.038875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8d3fb-536c-402b-8403-16bd31a10417 admin admin] PATH_INFO: `/v3/projects/6f889848d459459484b5b5517fef576b/users/289c7d8b89cd445a924f5ac07e85fe48/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.043318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 797/1606] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:26 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:27.043318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e90200-c5a1-4ddc-ae49-938e9f748874 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.057144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8d3fb-536c-402b-8403-16bd31a10417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f889848d459459484b5b5517fef576b, user_id=289c7d8b89cd445a924f5ac07e85fe48, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.059899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8d3fb-536c-402b-8403-16bd31a10417 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.076546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 810/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:27 2022] PUT /identity/v3/projects/6f889848d459459484b5b5517fef576b/users/289c7d8b89cd445a924f5ac07e85fe48/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:27.077812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8d3fb-536c-402b-8403-16bd31a10417 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.100483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e90200-c5a1-4ddc-ae49-938e9f748874 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.140084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8d3fb-536c-402b-8403-16bd31a10417 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.144854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7a5e-8925-4b0f-9458-78c2f4ee0556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.145538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7a5e-8925-4b0f-9458-78c2f4ee0556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.145819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7a5e-8925-4b0f-9458-78c2f4ee0556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.146093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7a5e-8925-4b0f-9458-78c2f4ee0556 admin admin] PATH_INFO: `/v3/projects/d8eda7c7d6a348e08e81a6b2496b09e5/users/5573338f0bd64c729c7d89345069ca6f/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.151314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae426-7ebf-48dd-aadc-8e251c0c3f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.151920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae426-7ebf-48dd-aadc-8e251c0c3f24 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.152460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae426-7ebf-48dd-aadc-8e251c0c3f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.152725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae426-7ebf-48dd-aadc-8e251c0c3f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.160764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7a5e-8925-4b0f-9458-78c2f4ee0556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8eda7c7d6a348e08e81a6b2496b09e5, user_id=5573338f0bd64c729c7d89345069ca6f, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.162996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ae426-7ebf-48dd-aadc-8e251c0c3f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.163511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7a5e-8925-4b0f-9458-78c2f4ee0556 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.164412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ae426-7ebf-48dd-aadc-8e251c0c3f24 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.174408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 798/1608] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:27.175199 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae426-7ebf-48dd-aadc-8e251c0c3f24 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.181094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 811/1609] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:27 2022] PUT /identity/v3/projects/d8eda7c7d6a348e08e81a6b2496b09e5/users/5573338f0bd64c729c7d89345069ca6f/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:27.181094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7a5e-8925-4b0f-9458-78c2f4ee0556 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.186373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae426-7ebf-48dd-aadc-8e251c0c3f24 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.249842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7a5e-8925-4b0f-9458-78c2f4ee0556 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.250270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e0984-b06f-49ec-b9e3-05f296679d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.250899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e0984-b06f-49ec-b9e3-05f296679d06 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.251175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e0984-b06f-49ec-b9e3-05f296679d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.251442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e0984-b06f-49ec-b9e3-05f296679d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.255126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f423f6-55e3-450e-9ce1-af856a26720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.255842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f423f6-55e3-450e-9ce1-af856a26720a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.256293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f423f6-55e3-450e-9ce1-af856a26720a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.256473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f423f6-55e3-450e-9ce1-af856a26720a admin admin] PATH_INFO: `/v3/projects/65cecda1373f46d9af7eccf39e3cf05c/users/99d6cfb0cfdc4900b566880e1fe02a85/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.263790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e0984-b06f-49ec-b9e3-05f296679d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.265216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e0984-b06f-49ec-b9e3-05f296679d06 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.277172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f423f6-55e3-450e-9ce1-af856a26720a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cecda1373f46d9af7eccf39e3cf05c, user_id=99d6cfb0cfdc4900b566880e1fe02a85, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.280056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f423f6-55e3-450e-9ce1-af856a26720a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.281343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 799/1610] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:27.283022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bdc0d-a1aa-469c-b3bc-3dd2f4673535 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.283414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bdc0d-a1aa-469c-b3bc-3dd2f4673535 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.283773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bdc0d-a1aa-469c-b3bc-3dd2f4673535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.296731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 812/1611] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:27 2022] PUT /identity/v3/projects/65cecda1373f46d9af7eccf39e3cf05c/users/99d6cfb0cfdc4900b566880e1fe02a85/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:27.307173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f423f6-55e3-450e-9ce1-af856a26720a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.316425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ca9bdc0d-a1aa-469c-b3bc-3dd2f4673535 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:27.317834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ca9bdc0d-a1aa-469c-b3bc-3dd2f4673535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:27.366291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:27.380268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:27.391718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f423f6-55e3-450e-9ce1-af856a26720a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.398394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b212e99-88f8-49cb-a0ef-891928f4129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.399105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b212e99-88f8-49cb-a0ef-891928f4129d admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.399894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b212e99-88f8-49cb-a0ef-891928f4129d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.400234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b212e99-88f8-49cb-a0ef-891928f4129d admin admin] PATH_INFO: `/v3/projects/40ec5f56eca443eea502ea4afeadeaf8/users/c01b6746d6024e899578fb9180daf3d5/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.411684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 800/1612] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:27.411684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bdc0d-a1aa-469c-b3bc-3dd2f4673535 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.417590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b212e99-88f8-49cb-a0ef-891928f4129d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ec5f56eca443eea502ea4afeadeaf8, user_id=c01b6746d6024e899578fb9180daf3d5, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.420257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b212e99-88f8-49cb-a0ef-891928f4129d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.421411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bdc0d-a1aa-469c-b3bc-3dd2f4673535 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.427037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fe32d-b94e-43d4-a8d9-57caf0784789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.427823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fe32d-b94e-43d4-a8d9-57caf0784789 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.428325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fe32d-b94e-43d4-a8d9-57caf0784789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.428748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fe32d-b94e-43d4-a8d9-57caf0784789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.429489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fe32d-b94e-43d4-a8d9-57caf0784789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.431762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fe32d-b94e-43d4-a8d9-57caf0784789 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.440779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 813/1613] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:27 2022] PUT /identity/v3/projects/40ec5f56eca443eea502ea4afeadeaf8/users/c01b6746d6024e899578fb9180daf3d5/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:27.443036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c75c-4315-48b5-b115-1a29396bfa93 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.443630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c75c-4315-48b5-b115-1a29396bfa93 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.444062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c75c-4315-48b5-b115-1a29396bfa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.454271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 801/1614] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:27.456427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028ebd-6d7d-4016-807b-ce44e02d3091 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.456952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028ebd-6d7d-4016-807b-ce44e02d3091 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.457610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028ebd-6d7d-4016-807b-ce44e02d3091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.473975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3144c75c-4315-48b5-b115-1a29396bfa93 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:27.475210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3144c75c-4315-48b5-b115-1a29396bfa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:27.483289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b3028ebd-6d7d-4016-807b-ce44e02d3091 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:27.485543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b3028ebd-6d7d-4016-807b-ce44e02d3091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:27.562995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 814/1615] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:27.565420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcd5ad-b6d4-48c3-a78e-d6ff3c88a9d1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.565817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcd5ad-b6d4-48c3-a78e-d6ff3c88a9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.566808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcd5ad-b6d4-48c3-a78e-d6ff3c88a9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.579006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 802/1616] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:27.579981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3028ebd-6d7d-4016-807b-ce44e02d3091 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.605720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-58bcd5ad-b6d4-48c3-a78e-d6ff3c88a9d1 None None] MFA Rules not processed for user `99d6cfb0cfdc4900b566880e1fe02a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:27.606971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-58bcd5ad-b6d4-48c3-a78e-d6ff3c88a9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:27.657308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3028ebd-6d7d-4016-807b-ce44e02d3091 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.662317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ac6d1-afe9-4ff1-9068-61c1eea644dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.663429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ac6d1-afe9-4ff1-9068-61c1eea644dc service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.663987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ac6d1-afe9-4ff1-9068-61c1eea644dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.664285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ac6d1-afe9-4ff1-9068-61c1eea644dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.677165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ac6d1-afe9-4ff1-9068-61c1eea644dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.678677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ac6d1-afe9-4ff1-9068-61c1eea644dc service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.690070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 803/1617] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:27.691616 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d3e4-dc37-4913-a7ed-f8453d4a8ba4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.692075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d3e4-dc37-4913-a7ed-f8453d4a8ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.692506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d3e4-dc37-4913-a7ed-f8453d4a8ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.703890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 815/1618] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:27.708287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcd5ad-b6d4-48c3-a78e-d6ff3c88a9d1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.720092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcd5ad-b6d4-48c3-a78e-d6ff3c88a9d1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.722655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d543d3e4-dc37-4913-a7ed-f8453d4a8ba4 None None] MFA Rules not processed for user `c01b6746d6024e899578fb9180daf3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:27.723871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d543d3e4-dc37-4913-a7ed-f8453d4a8ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:27.725374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913a795-9be7-4e3a-a937-cbd35986e27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.726064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913a795-9be7-4e3a-a937-cbd35986e27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.726377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913a795-9be7-4e3a-a937-cbd35986e27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.726601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913a795-9be7-4e3a-a937-cbd35986e27f admin admin] PATH_INFO: `/v3/projects/c1b7b2437b6947b6a71cd952a28ecfeb/users/bc7f8d89f4794a71a9ed90b6cd3e4de4/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.748867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913a795-9be7-4e3a-a937-cbd35986e27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b7b2437b6947b6a71cd952a28ecfeb, user_id=bc7f8d89f4794a71a9ed90b6cd3e4de4, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.752224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913a795-9be7-4e3a-a937-cbd35986e27f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.771289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 816/1619] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:27 2022] PUT /identity/v3/projects/c1b7b2437b6947b6a71cd952a28ecfeb/users/bc7f8d89f4794a71a9ed90b6cd3e4de4/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:27.772141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913a795-9be7-4e3a-a937-cbd35986e27f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.822420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:27.823127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 804/1620] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:27.824022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543d3e4-dc37-4913-a7ed-f8453d4a8ba4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.840434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543d3e4-dc37-4913-a7ed-f8453d4a8ba4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.845681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72649a48-aa56-4aa7-8f17-60b8f987ccbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.846392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72649a48-aa56-4aa7-8f17-60b8f987ccbc service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.846741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72649a48-aa56-4aa7-8f17-60b8f987ccbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.847489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72649a48-aa56-4aa7-8f17-60b8f987ccbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.851938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913a795-9be7-4e3a-a937-cbd35986e27f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.858012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e17f2f-2338-47f4-9bcf-ed0f2b2384e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:27.858859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e17f2f-2338-47f4-9bcf-ed0f2b2384e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:27.859342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e17f2f-2338-47f4-9bcf-ed0f2b2384e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:27.859740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e17f2f-2338-47f4-9bcf-ed0f2b2384e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:27.859882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:27.961814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e17f2f-2338-47f4-9bcf-ed0f2b2384e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.964116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72649a48-aa56-4aa7-8f17-60b8f987ccbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:27.964858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e17f2f-2338-47f4-9bcf-ed0f2b2384e5 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.965944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72649a48-aa56-4aa7-8f17-60b8f987ccbc service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:27.978483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 817/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:27.979611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e17f2f-2338-47f4-9bcf-ed0f2b2384e5 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.980696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 805/1622] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:27.985367 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72649a48-aa56-4aa7-8f17-60b8f987ccbc service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:27.997613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72649a48-aa56-4aa7-8f17-60b8f987ccbc service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:27.998556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e17f2f-2338-47f4-9bcf-ed0f2b2384e5 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.005179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5cc2d-318c-4708-be6c-eb420ab5d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.005886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5cc2d-318c-4708-be6c-eb420ab5d60d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.006319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5cc2d-318c-4708-be6c-eb420ab5d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.006593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5cc2d-318c-4708-be6c-eb420ab5d60d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.007537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5cc2d-318c-4708-be6c-eb420ab5d60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.010009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5cc2d-318c-4708-be6c-eb420ab5d60d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.014290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375eaa8-c32e-47af-bdd2-c31b772f5dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.016847 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375eaa8-c32e-47af-bdd2-c31b772f5dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.017130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375eaa8-c32e-47af-bdd2-c31b772f5dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.018000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375eaa8-c32e-47af-bdd2-c31b772f5dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.020030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375eaa8-c32e-47af-bdd2-c31b772f5dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.025828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 806/1623] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:28.027552 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bb5a1-0798-4d31-b7ea-ca119402b8a2 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.027835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bb5a1-0798-4d31-b7ea-ca119402b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.028405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bb5a1-0798-4d31-b7ea-ca119402b8a2 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.029583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375eaa8-c32e-47af-bdd2-c31b772f5dc4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.037622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 807/1624] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 10 13:37:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:37:28.039233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bb5a1-0798-4d31-b7ea-ca119402b8a2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:28.063908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bb5a1-0798-4d31-b7ea-ca119402b8a2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.073836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 818/1625] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:27 2022] POST /identity/v3/projects => generated 405 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:28.075514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247af90-ead4-4ae2-9c6c-97573a3474c6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.075801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247af90-ead4-4ae2-9c6c-97573a3474c6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.076250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247af90-ead4-4ae2-9c6c-97573a3474c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.081076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a1a9-360d-4da7-9561-df058c576e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.081904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a1a9-360d-4da7-9561-df058c576e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.082195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a1a9-360d-4da7-9561-df058c576e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.082524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a1a9-360d-4da7-9561-df058c576e2d admin admin] PATH_INFO: `/v3/projects/c1b7b2437b6947b6a71cd952a28ecfeb/users/bc7f8d89f4794a71a9ed90b6cd3e4de4/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.084929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665a1a9-360d-4da7-9561-df058c576e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b7b2437b6947b6a71cd952a28ecfeb, user_id=bc7f8d89f4794a71a9ed90b6cd3e4de4, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.090425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665a1a9-360d-4da7-9561-df058c576e2d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.114374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 808/1626] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:28 2022] PUT /identity/v3/projects/c1b7b2437b6947b6a71cd952a28ecfeb/users/bc7f8d89f4794a71a9ed90b6cd3e4de4/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:28.114374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a1a9-360d-4da7-9561-df058c576e2d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:28.145310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9247af90-ead4-4ae2-9c6c-97573a3474c6 None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:28.147592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9247af90-ead4-4ae2-9c6c-97573a3474c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:28.202186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a1a9-360d-4da7-9561-df058c576e2d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.208301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a57c6-7877-4829-ba48-0f1892f21954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.209476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a57c6-7877-4829-ba48-0f1892f21954 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.209806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a57c6-7877-4829-ba48-0f1892f21954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.210242 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a57c6-7877-4829-ba48-0f1892f21954 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.211220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a57c6-7877-4829-ba48-0f1892f21954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.213099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a57c6-7877-4829-ba48-0f1892f21954 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.255228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 809/1627] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:28 2022] POST /identity/v3/users => generated 309 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:28.257576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a57c6-7877-4829-ba48-0f1892f21954 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:28.263954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 819/1628] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 10 13:37:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:28.279147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247af90-ead4-4ae2-9c6c-97573a3474c6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:28.284786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a57c6-7877-4829-ba48-0f1892f21954 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.298836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cef53-adb8-4635-8879-dfe94c217407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.299687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cef53-adb8-4635-8879-dfe94c217407 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.299965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cef53-adb8-4635-8879-dfe94c217407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.300661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247af90-ead4-4ae2-9c6c-97573a3474c6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.301928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cef53-adb8-4635-8879-dfe94c217407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.303337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cef53-adb8-4635-8879-dfe94c217407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.307186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cef53-adb8-4635-8879-dfe94c217407 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.315783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590475e-2861-4d7c-af7b-35e49330a24a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.316491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590475e-2861-4d7c-af7b-35e49330a24a service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.316770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590475e-2861-4d7c-af7b-35e49330a24a service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.317045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590475e-2861-4d7c-af7b-35e49330a24a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.334657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 810/1629] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:28.339278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cef53-adb8-4635-8879-dfe94c217407 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:28.369359 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cef53-adb8-4635-8879-dfe94c217407 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.382723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42013bf-01f8-482f-a893-da91210d344f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.383410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42013bf-01f8-482f-a893-da91210d344f admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.383729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42013bf-01f8-482f-a893-da91210d344f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.384001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42013bf-01f8-482f-a893-da91210d344f admin admin] PATH_INFO: `/v3/projects/4d0d4351817147ebb712386d3bd05f7a/users/668cfdacde9a4c15aa432cd7cf199cfd/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.421157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42013bf-01f8-482f-a893-da91210d344f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0d4351817147ebb712386d3bd05f7a, user_id=668cfdacde9a4c15aa432cd7cf199cfd, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.424062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42013bf-01f8-482f-a893-da91210d344f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.434600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:28.446319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 811/1630] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:28 2022] PUT /identity/v3/projects/4d0d4351817147ebb712386d3bd05f7a/users/668cfdacde9a4c15aa432cd7cf199cfd/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:28.450374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42013bf-01f8-482f-a893-da91210d344f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:28.468060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590475e-2861-4d7c-af7b-35e49330a24a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.480017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590475e-2861-4d7c-af7b-35e49330a24a service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.503431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:28.508060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 820/1631] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:28.536471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42013bf-01f8-482f-a893-da91210d344f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.543017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a319a77-1fb4-4f41-a195-5891a7c3c146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.544091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319a77-1fb4-4f41-a195-5891a7c3c146 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.544377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319a77-1fb4-4f41-a195-5891a7c3c146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.544648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319a77-1fb4-4f41-a195-5891a7c3c146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.545435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a319a77-1fb4-4f41-a195-5891a7c3c146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.547016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a319a77-1fb4-4f41-a195-5891a7c3c146 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.559925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 812/1632] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:37:28.563745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590475e-2861-4d7c-af7b-35e49330a24a service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:28.576871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590475e-2861-4d7c-af7b-35e49330a24a service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.588497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bbab-98c5-4ad5-8629-dad5bef17ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.589572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bbab-98c5-4ad5-8629-dad5bef17ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.590618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bbab-98c5-4ad5-8629-dad5bef17ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.590618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bbab-98c5-4ad5-8629-dad5bef17ae6 admin admin] PATH_INFO: `/v3/projects/4d0d4351817147ebb712386d3bd05f7a/users/668cfdacde9a4c15aa432cd7cf199cfd/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.597427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5bbab-98c5-4ad5-8629-dad5bef17ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0d4351817147ebb712386d3bd05f7a, user_id=668cfdacde9a4c15aa432cd7cf199cfd, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:28.605506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5bbab-98c5-4ad5-8629-dad5bef17ae6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:28.634482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 821/1633] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:28 2022] PUT /identity/v3/projects/4d0d4351817147ebb712386d3bd05f7a/users/668cfdacde9a4c15aa432cd7cf199cfd/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:28.644884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6edea-8c79-400b-86d3-7efbb744b592 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.645569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6edea-8c79-400b-86d3-7efbb744b592 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.647231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6edea-8c79-400b-86d3-7efbb744b592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:28.711680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a1c6edea-8c79-400b-86d3-7efbb744b592 None None] MFA Rules not processed for user `668cfdacde9a4c15aa432cd7cf199cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:28.714136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a1c6edea-8c79-400b-86d3-7efbb744b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:28.853134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 813/1634] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:28.867721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bbab-98c5-4ad5-8629-dad5bef17ae6 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:28.976525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bbab-98c5-4ad5-8629-dad5bef17ae6 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:28.990763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db528fd-569d-41dc-81fe-1104911c1029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:28.990763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db528fd-569d-41dc-81fe-1104911c1029 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:28.990763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db528fd-569d-41dc-81fe-1104911c1029 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:28.990763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db528fd-569d-41dc-81fe-1104911c1029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:29.008254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db528fd-569d-41dc-81fe-1104911c1029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:29.010318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db528fd-569d-41dc-81fe-1104911c1029 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:29.040893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 822/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:28 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:29.410989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6edea-8c79-400b-86d3-7efbb744b592 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:29.583054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6edea-8c79-400b-86d3-7efbb744b592 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:29.589618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df8c4-5a88-40c5-9e08-c287c60a188c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:29.590588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df8c4-5a88-40c5-9e08-c287c60a188c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:29.591078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df8c4-5a88-40c5-9e08-c287c60a188c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:29.593407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 823/1636] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 10 13:37:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:37:29.604590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:29.605503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:29.605777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:29.606051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] PATH_INFO: `/v3/users/99d6cfb0cfdc4900b566880e1fe02a85` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:29.607077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0602247-fbef-4d2d-b194-527d181776bb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:29.607522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0602247-fbef-4d2d-b194-527d181776bb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:29.607888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d6cfb0cfdc4900b566880e1fe02a85)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:29.608096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0602247-fbef-4d2d-b194-527d181776bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:29.610025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:29.646632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c0602247-fbef-4d2d-b194-527d181776bb None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:29.648931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c0602247-fbef-4d2d-b194-527d181776bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:29.723044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d6cfb0cfdc4900b566880e1fe02a85'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:29.745545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d6cfb0cfdc4900b566880e1fe02a85'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:29.746416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d6cfb0cfdc4900b566880e1fe02a85'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:29.749690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:29.757335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d6cfb0cfdc4900b566880e1fe02a85'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:29.792742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-007c479e-43b7-4634-b3e7-49eebbfa5da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d6cfb0cfdc4900b566880e1fe02a85'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:29.806489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 814/1637] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:29 2022] DELETE /identity/v3/users/99d6cfb0cfdc4900b566880e1fe02a85 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:29.811812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4ca62-43ce-49bc-a15f-6c86d2143c8b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:29.813019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4ca62-43ce-49bc-a15f-6c86d2143c8b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:29.813395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4ca62-43ce-49bc-a15f-6c86d2143c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:29.817310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 824/1638] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 10 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:29.824025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d8445-064b-40ad-902d-a65981544d64 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:29.824397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d8445-064b-40ad-902d-a65981544d64 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:29.824740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d8445-064b-40ad-902d-a65981544d64 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:29.826687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 825/1639] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 10 13:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:37:29.831699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ac30c-5bf1-4939-a49e-edb5f509256a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:29.832326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ac30c-5bf1-4939-a49e-edb5f509256a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:29.832705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ac30c-5bf1-4939-a49e-edb5f509256a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:29.860438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cba4ca62-43ce-49bc-a15f-6c86d2143c8b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:29.861943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cba4ca62-43ce-49bc-a15f-6c86d2143c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:29.876174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-432ac30c-5bf1-4939-a49e-edb5f509256a None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:29.877689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-432ac30c-5bf1-4939-a49e-edb5f509256a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:30.017442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:30.020421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 815/1640] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:30.038358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 826/1641] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:30.053399 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4ca62-43ce-49bc-a15f-6c86d2143c8b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:30.055324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cba4ca62-43ce-49bc-a15f-6c86d2143c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:30.111050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ac30c-5bf1-4939-a49e-edb5f509256a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:30.113127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-432ac30c-5bf1-4939-a49e-edb5f509256a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:30.170555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4ca62-43ce-49bc-a15f-6c86d2143c8b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:30.178607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8da89-7346-4602-8158-281e082c64ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:30.181021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8da89-7346-4602-8158-281e082c64ea service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:30.182129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8da89-7346-4602-8158-281e082c64ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:30.183176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8da89-7346-4602-8158-281e082c64ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:30.211305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8da89-7346-4602-8158-281e082c64ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:30.213954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8da89-7346-4602-8158-281e082c64ea service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:30.242509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 816/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:30.306701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ac30c-5bf1-4939-a49e-edb5f509256a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:30.316624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c762b15-bbf7-4cae-998a-b343d0d60580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:30.317328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c762b15-bbf7-4cae-998a-b343d0d60580 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:30.318014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c762b15-bbf7-4cae-998a-b343d0d60580 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:30.318014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c762b15-bbf7-4cae-998a-b343d0d60580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:30.338698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c762b15-bbf7-4cae-998a-b343d0d60580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:30.340648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c762b15-bbf7-4cae-998a-b343d0d60580 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:30.365839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 827/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:30.453961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23adb1a-9fcd-4e21-993d-5e29f0eb03ba None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:30.454167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23adb1a-9fcd-4e21-993d-5e29f0eb03ba None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:30.454522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23adb1a-9fcd-4e21-993d-5e29f0eb03ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:30.499361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3337-9409-4437-97fc-086a6455768c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:30.499544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3337-9409-4437-97fc-086a6455768c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:30.499818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3337-9409-4437-97fc-086a6455768c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:30.501600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 828/1644] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 10 13:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:37:30.507206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20eb74-78cc-4a8b-bb95-0ea46086c007 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:30.507478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20eb74-78cc-4a8b-bb95-0ea46086c007 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:30.507749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20eb74-78cc-4a8b-bb95-0ea46086c007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:30.515701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a23adb1a-9fcd-4e21-993d-5e29f0eb03ba None None] MFA Rules not processed for user `bc7f8d89f4794a71a9ed90b6cd3e4de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:30.517625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a23adb1a-9fcd-4e21-993d-5e29f0eb03ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:30.569954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2a20eb74-78cc-4a8b-bb95-0ea46086c007 None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:30.571228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2a20eb74-78cc-4a8b-bb95-0ea46086c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:30.596042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 829/1645] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:37:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:30.611324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20eb74-78cc-4a8b-bb95-0ea46086c007 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:30.612397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 817/1646] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:30.613222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2a20eb74-78cc-4a8b-bb95-0ea46086c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:30.625966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20eb74-78cc-4a8b-bb95-0ea46086c007 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:30.631724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822571c-4ccd-42a3-b993-4f5095702704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:30.632733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822571c-4ccd-42a3-b993-4f5095702704 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:30.633133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822571c-4ccd-42a3-b993-4f5095702704 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:30.633601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822571c-4ccd-42a3-b993-4f5095702704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:30.648742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822571c-4ccd-42a3-b993-4f5095702704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:30.652406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822571c-4ccd-42a3-b993-4f5095702704 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:30.668245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 830/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:30.843344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23adb1a-9fcd-4e21-993d-5e29f0eb03ba None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:30.845016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a23adb1a-9fcd-4e21-993d-5e29f0eb03ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:30.872215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:30.882752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23adb1a-9fcd-4e21-993d-5e29f0eb03ba None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:30.891613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:30.892880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:30.893431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:30.893968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] PATH_INFO: `/v3/users/c01b6746d6024e899578fb9180daf3d5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:30.895593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c01b6746d6024e899578fb9180daf3d5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:30.898062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:31.015717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c01b6746d6024e899578fb9180daf3d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:31.043881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c01b6746d6024e899578fb9180daf3d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:31.044856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c01b6746d6024e899578fb9180daf3d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:31.059100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c01b6746d6024e899578fb9180daf3d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:31.096056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c01b6746d6024e899578fb9180daf3d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:31.107867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 818/1648] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:30 2022] DELETE /identity/v3/users/c01b6746d6024e899578fb9180daf3d5 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:31.109590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d11d9-738b-49a2-8c0a-91600025d807 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:31.109824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d11d9-738b-49a2-8c0a-91600025d807 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:31.110099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d11d9-738b-49a2-8c0a-91600025d807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:31.142685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d15d11d9-738b-49a2-8c0a-91600025d807 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:31.208230 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d15d11d9-738b-49a2-8c0a-91600025d807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:31.297004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 831/1649] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:31.324711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:31.326707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:31.431087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2785b0-1a05-4618-813c-5ac153e664ce admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:31.440951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba64230-f4eb-43a6-bbfd-9a657c6643ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:31.441955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba64230-f4eb-43a6-bbfd-9a657c6643ba service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:31.442477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba64230-f4eb-43a6-bbfd-9a657c6643ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:31.443025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba64230-f4eb-43a6-bbfd-9a657c6643ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:31.456718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba64230-f4eb-43a6-bbfd-9a657c6643ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:31.458838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba64230-f4eb-43a6-bbfd-9a657c6643ba service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:31.476353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 819/1650] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:31.908385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290424-ae20-48c3-af5b-ce73614ae45c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:31.909286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290424-ae20-48c3-af5b-ce73614ae45c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:31.909746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290424-ae20-48c3-af5b-ce73614ae45c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:31.911568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 832/1651] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:37:32.012381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a846a-9ab8-441b-ba05-6f96e0035b5e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:32.012933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a846a-9ab8-441b-ba05-6f96e0035b5e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:32.012933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a846a-9ab8-441b-ba05-6f96e0035b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:32.060470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0d5a846a-9ab8-441b-ba05-6f96e0035b5e None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:32.064692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0d5a846a-9ab8-441b-ba05-6f96e0035b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:32.221168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 820/1652] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:37:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:32.262715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8290424-ae20-48c3-af5b-ce73614ae45c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:32.264480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f8290424-ae20-48c3-af5b-ce73614ae45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:32.303296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8290424-ae20-48c3-af5b-ce73614ae45c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:32.315342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b621a-941e-4791-9f3b-1872523e64fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:32.315967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b621a-941e-4791-9f3b-1872523e64fe service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:32.316247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b621a-941e-4791-9f3b-1872523e64fe service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:32.316513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b621a-941e-4791-9f3b-1872523e64fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:32.338242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b621a-941e-4791-9f3b-1872523e64fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:32.350554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b621a-941e-4791-9f3b-1872523e64fe service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:32.391269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 833/1653] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:32.742365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a846a-9ab8-441b-ba05-6f96e0035b5e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:32.744012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0d5a846a-9ab8-441b-ba05-6f96e0035b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:32.768258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a846a-9ab8-441b-ba05-6f96e0035b5e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:32.780077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:32.781251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:32.781251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:32.781486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] PATH_INFO: `/v3/projects/65cecda1373f46d9af7eccf39e3cf05c` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:32.787000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cecda1373f46d9af7eccf39e3cf05c)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:32.788135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:37:32.788135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Aug 10 13:37:32.788135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: Aug 10 13:37:32.789504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:32.808482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:32.889673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cecda1373f46d9af7eccf39e3cf05c', 'request_id': 'req-635ef8e2-9ce1-4795-b427-852a0fcb4194'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:32.898069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cecda1373f46d9af7eccf39e3cf05c', 'request_id': 'req-635ef8e2-9ce1-4795-b427-852a0fcb4194'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:32.978324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cecda1373f46d9af7eccf39e3cf05c', 'request_id': 'req-635ef8e2-9ce1-4795-b427-852a0fcb4194'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:33.016734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] The token cache is being invalidate because project 65cecda1373f46d9af7eccf39e3cf05c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:33.017351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:33.017842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:33.019619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 821/1654] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:32 2022] DELETE /identity/v3/projects/65cecda1373f46d9af7eccf39e3cf05c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:33.023540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b621a-941e-4791-9f3b-1872523e64fe service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:33.041888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b621a-941e-4791-9f3b-1872523e64fe service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:33.183119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:33.184093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:33.184642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:33.185092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] PATH_INFO: `/v3/users/668cfdacde9a4c15aa432cd7cf199cfd` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:33.186703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=668cfdacde9a4c15aa432cd7cf199cfd)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:33.188085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 13:37:33.188085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Aug 10 13:37:33.188085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: Aug 10 13:37:33.190533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:33.201712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:33.373141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668cfdacde9a4c15aa432cd7cf199cfd'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:33.393893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668cfdacde9a4c15aa432cd7cf199cfd'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:33.394895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668cfdacde9a4c15aa432cd7cf199cfd'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:33.437983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668cfdacde9a4c15aa432cd7cf199cfd'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:33.438811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dd783e3b-448b-49a3-bc10-e2204bace852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668cfdacde9a4c15aa432cd7cf199cfd'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:33.470369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 834/1655] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:33 2022] DELETE /identity/v3/users/668cfdacde9a4c15aa432cd7cf199cfd => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:33.941506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:33.942993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:33.966843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ef8e2-9ce1-4795-b427-852a0fcb4194 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:34.084180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:34.084888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.085119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.085568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] PATH_INFO: `/v3/projects/40ec5f56eca443eea502ea4afeadeaf8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.087505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ec5f56eca443eea502ea4afeadeaf8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:34.090198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:34.200218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ec5f56eca443eea502ea4afeadeaf8', 'request_id': 'req-7f864f19-7076-4723-a29a-47bcd10d9644'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.213450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ec5f56eca443eea502ea4afeadeaf8', 'request_id': 'req-7f864f19-7076-4723-a29a-47bcd10d9644'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.226361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ec5f56eca443eea502ea4afeadeaf8', 'request_id': 'req-7f864f19-7076-4723-a29a-47bcd10d9644'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.292278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] The token cache is being invalidate because project 40ec5f56eca443eea502ea4afeadeaf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:34.292863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.294017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.299691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 822/1656] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:33 2022] DELETE /identity/v3/projects/40ec5f56eca443eea502ea4afeadeaf8 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:34.301135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489d0f-46bb-4c88-898f-06e26a3111f2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.301657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489d0f-46bb-4c88-898f-06e26a3111f2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.301731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489d0f-46bb-4c88-898f-06e26a3111f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.324467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:34.328016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:34.342496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ca489d0f-46bb-4c88-898f-06e26a3111f2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:34.345743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ca489d0f-46bb-4c88-898f-06e26a3111f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:34.395933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:34.472797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 835/1657] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:34.473063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f864f19-7076-4723-a29a-47bcd10d9644 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:34.481270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ff022-b31d-4ec6-8ec9-44e1b4555c1d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.481765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ff022-b31d-4ec6-8ec9-44e1b4555c1d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.482185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ff022-b31d-4ec6-8ec9-44e1b4555c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.484529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:34.485192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.485470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.485729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] PATH_INFO: `/v3/projects/4d0d4351817147ebb712386d3bd05f7a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.486852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0d4351817147ebb712386d3bd05f7a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:34.489045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:34.521557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c42ff022-b31d-4ec6-8ec9-44e1b4555c1d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:34.522776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c42ff022-b31d-4ec6-8ec9-44e1b4555c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:34.543905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 836/1658] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:34.547966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ff022-b31d-4ec6-8ec9-44e1b4555c1d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:34.553193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0d4351817147ebb712386d3bd05f7a', 'request_id': 'req-23e43c8e-e54b-441c-ab47-62cc07753ff2'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.558610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ff022-b31d-4ec6-8ec9-44e1b4555c1d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:34.559788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0d4351817147ebb712386d3bd05f7a', 'request_id': 'req-23e43c8e-e54b-441c-ab47-62cc07753ff2'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.568929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0d4351817147ebb712386d3bd05f7a', 'request_id': 'req-23e43c8e-e54b-441c-ab47-62cc07753ff2'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.595515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] The token cache is being invalidate because project 4d0d4351817147ebb712386d3bd05f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:34.596031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.597509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23e43c8e-e54b-441c-ab47-62cc07753ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:34.600878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 823/1659] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:34 2022] DELETE /identity/v3/projects/4d0d4351817147ebb712386d3bd05f7a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:37:34.606309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c88df8-1b60-4107-93da-d33cc780448c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.606711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c88df8-1b60-4107-93da-d33cc780448c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.607056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c88df8-1b60-4107-93da-d33cc780448c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.666806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-25c88df8-1b60-4107-93da-d33cc780448c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:34.671211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-25c88df8-1b60-4107-93da-d33cc780448c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:34.686785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:34.687901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.688380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.688850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.689730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:34.690235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:34.692067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:34.720860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 837/1660] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:37:34.724878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:34.729438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:34.768092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63630beb-ab26-4df5-88b0-b0de2645796c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:34.782024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:34.783590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.784538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.785401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.786391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:34.794449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:34.815695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:34.841749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 824/1661] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:34.844511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04ad70-e97b-47f6-8a4d-95d94543c8fc None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.845750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04ad70-e97b-47f6-8a4d-95d94543c8fc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.846036 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04ad70-e97b-47f6-8a4d-95d94543c8fc None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.849974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 825/1662] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:37:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:37:34.854263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85b86d-f4fc-4921-89cb-37d11bd0e3b3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:34.854550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85b86d-f4fc-4921-89cb-37d11bd0e3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:34.856271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85b86d-f4fc-4921-89cb-37d11bd0e3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:34.858378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 838/1663] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:34 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:34.859828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:34.863024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:34.905904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cc85b86d-f4fc-4921-89cb-37d11bd0e3b3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:34.909466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cc85b86d-f4fc-4921-89cb-37d11bd0e3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:35.046777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:35.059246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ba973-9176-476f-9b36-13eaf73b2c09 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.074138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 826/1664] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:35.075319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85b86d-f4fc-4921-89cb-37d11bd0e3b3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:35.088024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:35.088692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.089109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.089403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] PATH_INFO: `/v3/users/f87f0cfd08584af49ee60cd7ad115eb4` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:35.093602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87f0cfd08584af49ee60cd7ad115eb4)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:35.099968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:35.111611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85b86d-f4fc-4921-89cb-37d11bd0e3b3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.132234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14705a-64e5-451f-a0ea-51dcf52738db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:35.132947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14705a-64e5-451f-a0ea-51dcf52738db admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.133132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14705a-64e5-451f-a0ea-51dcf52738db admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.137863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14705a-64e5-451f-a0ea-51dcf52738db admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:35.142336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14705a-64e5-451f-a0ea-51dcf52738db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:35.157629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14705a-64e5-451f-a0ea-51dcf52738db admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:35.194384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87f0cfd08584af49ee60cd7ad115eb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:35.199634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 827/1665] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:35 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:35.201040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14705a-64e5-451f-a0ea-51dcf52738db admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:35.213723 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87f0cfd08584af49ee60cd7ad115eb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:35.214429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87f0cfd08584af49ee60cd7ad115eb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:35.216997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14705a-64e5-451f-a0ea-51dcf52738db admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.226082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f133119-5f56-4c2f-b47a-d1386122a723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:35.226635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f133119-5f56-4c2f-b47a-d1386122a723 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.226914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f133119-5f56-4c2f-b47a-d1386122a723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.227370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f133119-5f56-4c2f-b47a-d1386122a723 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:35.228140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f133119-5f56-4c2f-b47a-d1386122a723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:35.228448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f133119-5f56-4c2f-b47a-d1386122a723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:35.230211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f133119-5f56-4c2f-b47a-d1386122a723 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:35.263599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87f0cfd08584af49ee60cd7ad115eb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:35.266483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87f0cfd08584af49ee60cd7ad115eb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:35.273404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 828/1666] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:35.273678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f73f04-9510-440b-aea5-fc1e9dd609d5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.275343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f73f04-9510-440b-aea5-fc1e9dd609d5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.277258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f73f04-9510-440b-aea5-fc1e9dd609d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:35.312480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 839/1667] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:34 2022] DELETE /identity/v3/users/f87f0cfd08584af49ee60cd7ad115eb4 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:35.316282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:35.319224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-40f73f04-9510-440b-aea5-fc1e9dd609d5 None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:35.320773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:35.328206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-40f73f04-9510-440b-aea5-fc1e9dd609d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:35.573930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68134f5b-535c-43f8-823d-4181643eab71 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.601951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba65844-3429-4cad-97a9-874dae3ef313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:35.602620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba65844-3429-4cad-97a9-874dae3ef313 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.602974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba65844-3429-4cad-97a9-874dae3ef313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.603248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba65844-3429-4cad-97a9-874dae3ef313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:35.603948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba65844-3429-4cad-97a9-874dae3ef313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:35.605619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba65844-3429-4cad-97a9-874dae3ef313 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:35.616984 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 829/1668] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:37:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:35.619019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f73f04-9510-440b-aea5-fc1e9dd609d5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:35.638383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 840/1669] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:35.644287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba65844-3429-4cad-97a9-874dae3ef313 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:35.645706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-40f73f04-9510-440b-aea5-fc1e9dd609d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:35.665053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:35.677139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba65844-3429-4cad-97a9-874dae3ef313 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.686117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:35.687098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.687795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.688373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:35.689215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:35.699732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:35.701857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f73f04-9510-440b-aea5-fc1e9dd609d5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.726208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfff29-18b2-4278-8579-ba5c036b160f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:35.726851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfff29-18b2-4278-8579-ba5c036b160f admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.727166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfff29-18b2-4278-8579-ba5c036b160f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.728171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfff29-18b2-4278-8579-ba5c036b160f admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:35.728814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfff29-18b2-4278-8579-ba5c036b160f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:35.735260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfff29-18b2-4278-8579-ba5c036b160f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:35.738069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 841/1670] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:35.739174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:35.743148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:35.792275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588faddf-b301-4173-abd2-92146fa373bf admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.798862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 830/1671] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:35 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:35.798862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfff29-18b2-4278-8579-ba5c036b160f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:35.801703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a2225-78da-4988-acaf-7a52ceff1aee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:35.802404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a2225-78da-4988-acaf-7a52ceff1aee service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.802791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a2225-78da-4988-acaf-7a52ceff1aee service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.803140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a2225-78da-4988-acaf-7a52ceff1aee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:35.833126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a2225-78da-4988-acaf-7a52ceff1aee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:35.834940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a2225-78da-4988-acaf-7a52ceff1aee service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:35.877775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 842/1672] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:35.879141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a2225-78da-4988-acaf-7a52ceff1aee service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:35.900375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a2225-78da-4988-acaf-7a52ceff1aee service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.956562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfff29-18b2-4278-8579-ba5c036b160f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:35.971021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4667c-9f3f-469d-afe3-ee476bc4c142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:35.971930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4667c-9f3f-469d-afe3-ee476bc4c142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:35.972358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4667c-9f3f-469d-afe3-ee476bc4c142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:35.972760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4667c-9f3f-469d-afe3-ee476bc4c142 admin admin] PATH_INFO: `/v3/projects/dcabbb9bbd954ff38b77a6c30f5f2268/users/30663d726cb242cf990bca67ea4649ac/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.015572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4667c-9f3f-469d-afe3-ee476bc4c142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcabbb9bbd954ff38b77a6c30f5f2268, user_id=30663d726cb242cf990bca67ea4649ac, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:36.018668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4667c-9f3f-469d-afe3-ee476bc4c142 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:36.033484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c7a7a-c84c-482d-a7a6-a877492d9ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:36.034737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c7a7a-c84c-482d-a7a6-a877492d9ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.035121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c7a7a-c84c-482d-a7a6-a877492d9ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.036110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c7a7a-c84c-482d-a7a6-a877492d9ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.045398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c7a7a-c84c-482d-a7a6-a877492d9ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:36.048406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c7a7a-c84c-482d-a7a6-a877492d9ee0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:36.048723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 831/1673] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:35 2022] PUT /identity/v3/projects/dcabbb9bbd954ff38b77a6c30f5f2268/users/30663d726cb242cf990bca67ea4649ac/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:36.059110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47a3a3-414f-448f-91d9-4d45b341bfb1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.059595 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47a3a3-414f-448f-91d9-4d45b341bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.060134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47a3a3-414f-448f-91d9-4d45b341bfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.091883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 843/1674] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:35 2022] POST /identity/v3/users => generated 282 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:36.096171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c7a7a-c84c-482d-a7a6-a877492d9ee0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:36.114266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ec47a3a3-414f-448f-91d9-4d45b341bfb1 None None] MFA Rules not processed for user `30663d726cb242cf990bca67ea4649ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:36.116569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ec47a3a3-414f-448f-91d9-4d45b341bfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:36.120114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c7a7a-c84c-482d-a7a6-a877492d9ee0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:36.161573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:36.303357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 832/1675] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:36.305193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47a3a3-414f-448f-91d9-4d45b341bfb1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:36.334271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:36.335520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.335904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.336293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.337739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:36.339450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:36.356622 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 844/1676] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:36.357697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:36.359633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:36.375526 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68e29d-43d9-4136-9e60-e65d24103cf6 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:36.393372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:36.394544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.395078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.395728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] PATH_INFO: `/v3/projects/79721e96e7374f97bff559a2ce49aacb` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.405348 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79721e96e7374f97bff559a2ce49aacb)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:36.410485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:36.475802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47a3a3-414f-448f-91d9-4d45b341bfb1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:36.484259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41982442-e3e0-4d83-9e96-e0ad26a46b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:36.484898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41982442-e3e0-4d83-9e96-e0ad26a46b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.485177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41982442-e3e0-4d83-9e96-e0ad26a46b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.485477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41982442-e3e0-4d83-9e96-e0ad26a46b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.486111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41982442-e3e0-4d83-9e96-e0ad26a46b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:36.488307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41982442-e3e0-4d83-9e96-e0ad26a46b66 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:36.512914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79721e96e7374f97bff559a2ce49aacb', 'request_id': 'req-ad407e46-3f5b-4dc9-a4c2-3380c634c572'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:36.527413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79721e96e7374f97bff559a2ce49aacb', 'request_id': 'req-ad407e46-3f5b-4dc9-a4c2-3380c634c572'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:36.530756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:36.547742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79721e96e7374f97bff559a2ce49aacb', 'request_id': 'req-ad407e46-3f5b-4dc9-a4c2-3380c634c572'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:36.554441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 833/1677] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:36 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:36.554441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41982442-e3e0-4d83-9e96-e0ad26a46b66 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:36.595922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] The token cache is being invalidate because project 79721e96e7374f97bff559a2ce49aacb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:36.596228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:36.598801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:36.601174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 845/1678] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:36 2022] DELETE /identity/v3/projects/79721e96e7374f97bff559a2ce49aacb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:36.601981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:36.612666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:36.762726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41982442-e3e0-4d83-9e96-e0ad26a46b66 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:36.772620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:36.773269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.773564 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.773889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.775021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:36.776654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:36.818314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad407e46-3f5b-4dc9-a4c2-3380c634c572 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:36.823226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 834/1679] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:36.834049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:36.843945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:36.849376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:36.850506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.851567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.851567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.864363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:36.876838 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:36.913753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ed58-6843-46fe-aef5-ccc8d46b0e45 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:36.919372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:36.920046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.920362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.920625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] PATH_INFO: `/v3/users/bc7f8d89f4794a71a9ed90b6cd3e4de4` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:36.922164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7f8d89f4794a71a9ed90b6cd3e4de4)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:36.924101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:36.928485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 846/1680] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:36 2022] POST /identity/v3/users => generated 297 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:36.931536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:36.936288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:36.976749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bd0fa-cd63-483e-999d-eae4ae0a0bfb admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:36.989213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:36.992787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:36.994561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:36.994939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] PATH_INFO: `/v3/projects/1bc3596983554ce993c08879363c83b1/users/08d8e5ff2a9f448abfce6faa3e20162c/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.009880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:37.032812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc3596983554ce993c08879363c83b1, user_id=08d8e5ff2a9f448abfce6faa3e20162c, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:37.035603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:37.042562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7f8d89f4794a71a9ed90b6cd3e4de4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:37.064419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7f8d89f4794a71a9ed90b6cd3e4de4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:37.065189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7f8d89f4794a71a9ed90b6cd3e4de4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:37.066948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 847/1681] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:36 2022] PUT /identity/v3/projects/1bc3596983554ce993c08879363c83b1/users/08d8e5ff2a9f448abfce6faa3e20162c/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:37.067991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:37.078667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:37.092130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7f8d89f4794a71a9ed90b6cd3e4de4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:37.128401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6612b315-b6ad-4c90-8c33-5fde38bbbc94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7f8d89f4794a71a9ed90b6cd3e4de4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:37.146203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 835/1682] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:36 2022] DELETE /identity/v3/users/bc7f8d89f4794a71a9ed90b6cd3e4de4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:37.148209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e84be-fcb6-4f29-b3cb-96c986dc22ae None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.148500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e84be-fcb6-4f29-b3cb-96c986dc22ae None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.148773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e84be-fcb6-4f29-b3cb-96c986dc22ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.222255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-989e84be-fcb6-4f29-b3cb-96c986dc22ae None None] MFA Rules not processed for user `08d8e5ff2a9f448abfce6faa3e20162c`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:37.224521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-989e84be-fcb6-4f29-b3cb-96c986dc22ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:37.271888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059640b-7654-47bc-a710-c5e946486677 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:37.381567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ecd43-b7f8-4bb9-b690-fa598e9dd7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:37.382770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ecd43-b7f8-4bb9-b690-fa598e9dd7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.383066 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ecd43-b7f8-4bb9-b690-fa598e9dd7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.383366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ecd43-b7f8-4bb9-b690-fa598e9dd7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.384124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ecd43-b7f8-4bb9-b690-fa598e9dd7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:37.385701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ecd43-b7f8-4bb9-b690-fa598e9dd7e8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:37.392585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:37.413730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 848/1683] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:37.419290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ecd43-b7f8-4bb9-b690-fa598e9dd7e8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:37.424657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 836/1684] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:37.428235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e84be-fcb6-4f29-b3cb-96c986dc22ae None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:37.429930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-989e84be-fcb6-4f29-b3cb-96c986dc22ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:37.437565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ecd43-b7f8-4bb9-b690-fa598e9dd7e8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:37.453881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea01aa-5220-4ca4-a208-1785bfce9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:37.454635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea01aa-5220-4ca4-a208-1785bfce9f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.454912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea01aa-5220-4ca4-a208-1785bfce9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.455189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea01aa-5220-4ca4-a208-1785bfce9f53 admin admin] PATH_INFO: `/v3/projects/9e929e12019045b682178da8a644a0f4/users/301953bab3344be79804e96644ef460f/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.470854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e84be-fcb6-4f29-b3cb-96c986dc22ae None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:37.482116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348dae8-6515-469f-b11a-9a3ee2fa3153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:37.483643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348dae8-6515-469f-b11a-9a3ee2fa3153 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.484079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348dae8-6515-469f-b11a-9a3ee2fa3153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.484750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348dae8-6515-469f-b11a-9a3ee2fa3153 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.486379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348dae8-6515-469f-b11a-9a3ee2fa3153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:37.494115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea01aa-5220-4ca4-a208-1785bfce9f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e929e12019045b682178da8a644a0f4, user_id=301953bab3344be79804e96644ef460f, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:37.494491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348dae8-6515-469f-b11a-9a3ee2fa3153 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:37.496802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea01aa-5220-4ca4-a208-1785bfce9f53 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:37.521628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 849/1685] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:37 2022] PUT /identity/v3/projects/9e929e12019045b682178da8a644a0f4/users/301953bab3344be79804e96644ef460f/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:37.525975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea01aa-5220-4ca4-a208-1785bfce9f53 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:37.556661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 837/1686] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:37 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:37.557731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348dae8-6515-469f-b11a-9a3ee2fa3153 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:37.647465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea01aa-5220-4ca4-a208-1785bfce9f53 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:37.713180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c15837-af12-4ca9-85ae-6d429114a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:37.713999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c15837-af12-4ca9-85ae-6d429114a936 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.714193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c15837-af12-4ca9-85ae-6d429114a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.714422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c15837-af12-4ca9-85ae-6d429114a936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.715073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348dae8-6515-469f-b11a-9a3ee2fa3153 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:37.715180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c15837-af12-4ca9-85ae-6d429114a936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:37.716809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c15837-af12-4ca9-85ae-6d429114a936 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:37.733966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 850/1687] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:37.734682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c15837-af12-4ca9-85ae-6d429114a936 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:37.737515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12968b40-04c6-4d48-9830-ad128dd3f90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:37.741067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968b40-04c6-4d48-9830-ad128dd3f90d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.741525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968b40-04c6-4d48-9830-ad128dd3f90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.741946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968b40-04c6-4d48-9830-ad128dd3f90d admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.743041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12968b40-04c6-4d48-9830-ad128dd3f90d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:37.745719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c15837-af12-4ca9-85ae-6d429114a936 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:37.746111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12968b40-04c6-4d48-9830-ad128dd3f90d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:37.752794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb52fa-1a09-4516-815e-6c924ba24f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:37.753685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb52fa-1a09-4516-815e-6c924ba24f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.753974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb52fa-1a09-4516-815e-6c924ba24f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.754249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb52fa-1a09-4516-815e-6c924ba24f8d admin admin] PATH_INFO: `/v3/projects/9e929e12019045b682178da8a644a0f4/users/301953bab3344be79804e96644ef460f/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.756435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb52fa-1a09-4516-815e-6c924ba24f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e929e12019045b682178da8a644a0f4, user_id=301953bab3344be79804e96644ef460f, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:37.760283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb52fa-1a09-4516-815e-6c924ba24f8d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:37.777083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 851/1688] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:37 2022] PUT /identity/v3/projects/9e929e12019045b682178da8a644a0f4/users/301953bab3344be79804e96644ef460f/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:37.778323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 838/1689] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:37 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:37.782766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb52fa-1a09-4516-815e-6c924ba24f8d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:37.788325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea38f5-a75f-4178-9f8a-dc3710b20e20 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.788611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea38f5-a75f-4178-9f8a-dc3710b20e20 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.788907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea38f5-a75f-4178-9f8a-dc3710b20e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.799996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:37.829260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-eaea38f5-a75f-4178-9f8a-dc3710b20e20 None None] MFA Rules not processed for user `301953bab3344be79804e96644ef460f`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:37.830618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eaea38f5-a75f-4178-9f8a-dc3710b20e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:37.912735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb52fa-1a09-4516-815e-6c924ba24f8d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:37.918190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c654-9ce8-46da-8c9d-d0335cd8ea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:37.919357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c654-9ce8-46da-8c9d-d0335cd8ea3c admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:37.919751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c654-9ce8-46da-8c9d-d0335cd8ea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:37.920867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c654-9ce8-46da-8c9d-d0335cd8ea3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:37.921672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983c654-9ce8-46da-8c9d-d0335cd8ea3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:37.923541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983c654-9ce8-46da-8c9d-d0335cd8ea3c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:37.930970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:37.943767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 839/1690] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:37.952943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea38f5-a75f-4178-9f8a-dc3710b20e20 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:37.954662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eaea38f5-a75f-4178-9f8a-dc3710b20e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:37.966993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 852/1691] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:37:37.972250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c654-9ce8-46da-8c9d-d0335cd8ea3c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:38.001831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c654-9ce8-46da-8c9d-d0335cd8ea3c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:38.029270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e23ba-99f9-437b-ab32-3b6918dbca5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:38.030095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e23ba-99f9-437b-ab32-3b6918dbca5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.031669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e23ba-99f9-437b-ab32-3b6918dbca5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.032032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e23ba-99f9-437b-ab32-3b6918dbca5a admin admin] PATH_INFO: `/v3/projects/2b70196cc87c4d3a9b6d8601225aa63e/users/fd8e494e5edc4c72bc71e76fd783479f/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.069945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e23ba-99f9-437b-ab32-3b6918dbca5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b70196cc87c4d3a9b6d8601225aa63e, user_id=fd8e494e5edc4c72bc71e76fd783479f, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:38.073165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e23ba-99f9-437b-ab32-3b6918dbca5a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:38.099800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 853/1692] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:37 2022] PUT /identity/v3/projects/2b70196cc87c4d3a9b6d8601225aa63e/users/fd8e494e5edc4c72bc71e76fd783479f/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:38.104900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e23ba-99f9-437b-ab32-3b6918dbca5a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:38.121154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea38f5-a75f-4178-9f8a-dc3710b20e20 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:38.202818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e23ba-99f9-437b-ab32-3b6918dbca5a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:38.213190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:38.214050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.214337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.214609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.215762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:38.217056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:38.233366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 854/1693] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:38.234442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:38.237835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:38.254040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5766b-d30e-48dd-a8a2-731b49d5c482 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:38.257577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80f8a9-4ba4-41a4-88fd-12f4cb7c086d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:38.260376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:38.261608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f8a9-4ba4-41a4-88fd-12f4cb7c086d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.261727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.262096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.262378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] PATH_INFO: `/v3/projects/c1b7b2437b6947b6a71cd952a28ecfeb` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.263981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f8a9-4ba4-41a4-88fd-12f4cb7c086d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.264383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f8a9-4ba4-41a4-88fd-12f4cb7c086d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.265033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b7b2437b6947b6a71cd952a28ecfeb)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:38.267850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:38.397984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b7b2437b6947b6a71cd952a28ecfeb', 'request_id': 'req-2b39badf-746a-4907-9619-6e182229612b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:38.416791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b7b2437b6947b6a71cd952a28ecfeb', 'request_id': 'req-2b39badf-746a-4907-9619-6e182229612b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:38.437727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b7b2437b6947b6a71cd952a28ecfeb', 'request_id': 'req-2b39badf-746a-4907-9619-6e182229612b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:38.462859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] The token cache is being invalidate because project c1b7b2437b6947b6a71cd952a28ecfeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:38.465358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:38.466824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:38.472136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 855/1694] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:38 2022] DELETE /identity/v3/projects/c1b7b2437b6947b6a71cd952a28ecfeb => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:38.473023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:38.477781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:38.482240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:38.489184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80f8a9-4ba4-41a4-88fd-12f4cb7c086d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:38.490686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80f8a9-4ba4-41a4-88fd-12f4cb7c086d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:38.539048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 840/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:38.542319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cff2b-fbbc-4872-8691-ff6b497e14e5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.544366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cff2b-fbbc-4872-8691-ff6b497e14e5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.544661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cff2b-fbbc-4872-8691-ff6b497e14e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.625808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0e1cff2b-fbbc-4872-8691-ff6b497e14e5 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:38.628122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:38.633058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0e1cff2b-fbbc-4872-8691-ff6b497e14e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:38.639026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39badf-746a-4907-9619-6e182229612b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:38.658124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cddcc5-edb2-41b5-84e1-fc41900efb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:38.658124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cddcc5-edb2-41b5-84e1-fc41900efb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.658124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cddcc5-edb2-41b5-84e1-fc41900efb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.658124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cddcc5-edb2-41b5-84e1-fc41900efb94 admin admin] PATH_INFO: `/v3/projects/2b70196cc87c4d3a9b6d8601225aa63e/users/fd8e494e5edc4c72bc71e76fd783479f/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.661724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cddcc5-edb2-41b5-84e1-fc41900efb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b70196cc87c4d3a9b6d8601225aa63e, user_id=fd8e494e5edc4c72bc71e76fd783479f, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:38.675772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cddcc5-edb2-41b5-84e1-fc41900efb94 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:38.681082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 841/1696] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:38.689918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6909-fb19-4ff9-b6ed-231f4b5f7edb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.690209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6909-fb19-4ff9-b6ed-231f4b5f7edb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.690483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6909-fb19-4ff9-b6ed-231f4b5f7edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.699999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 856/1697] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:38 2022] PUT /identity/v3/projects/2b70196cc87c4d3a9b6d8601225aa63e/users/fd8e494e5edc4c72bc71e76fd783479f/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:38.710130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7ca34-0829-417b-a853-a993c8789592 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.710415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7ca34-0829-417b-a853-a993c8789592 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.710791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7ca34-0829-417b-a853-a993c8789592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.736686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d16f6909-fb19-4ff9-b6ed-231f4b5f7edb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:38.738397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d16f6909-fb19-4ff9-b6ed-231f4b5f7edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:38.808873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-34d7ca34-0829-417b-a853-a993c8789592 None None] MFA Rules not processed for user `fd8e494e5edc4c72bc71e76fd783479f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:38.810447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-34d7ca34-0829-417b-a853-a993c8789592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:38.887864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 842/1698] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:38.892241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6909-fb19-4ff9-b6ed-231f4b5f7edb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:38.909353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6909-fb19-4ff9-b6ed-231f4b5f7edb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:38.924996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:38.925854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:38.926131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:38.926398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:38.927106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:38.927400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:38.931678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:38.963052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 857/1699] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:38.965300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7ca34-0829-417b-a853-a993c8789592 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:38.968094 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 843/1700] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:37:38.980118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:38.985334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:38.989367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7ca34-0829-417b-a853-a993c8789592 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:39.000412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b571b-ec04-4b24-b855-44e3aff39f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:39.001045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b571b-ec04-4b24-b855-44e3aff39f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.001348 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b571b-ec04-4b24-b855-44e3aff39f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.001618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b571b-ec04-4b24-b855-44e3aff39f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.015013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b571b-ec04-4b24-b855-44e3aff39f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:39.015013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b571b-ec04-4b24-b855-44e3aff39f0e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:39.073990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 858/1701] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:38 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:39.075480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed723b4-949b-43e9-a8f0-745a3f6d7ade None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.075764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed723b4-949b-43e9-a8f0-745a3f6d7ade None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.076032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed723b4-949b-43e9-a8f0-745a3f6d7ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.103273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fb4e6-a31f-46c9-9402-164ffdc5d1ed admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:39.140302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3ed723b4-949b-43e9-a8f0-745a3f6d7ade None None] MFA Rules not processed for user `9cecae4323404bd29fe8c05a704de273`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:39.145818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3ed723b4-949b-43e9-a8f0-745a3f6d7ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:39.228319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:39.228943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.229217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.229509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.284068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 859/1702] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:39.285005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed723b4-949b-43e9-a8f0-745a3f6d7ade None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:39.348732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:39.377986 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:39.381549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:39.404225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:39.411625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 844/1703] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:39.412782 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:39.414170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:39.444073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed723b4-949b-43e9-a8f0-745a3f6d7ade None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:39.455820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:39.456449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.456724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.456987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.457645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:39.464710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:39.514448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 860/1704] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:39 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:39.515522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:39.518087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:39.550172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d440f1-38b3-4366-a118-fb3d50c0dc77 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:39.568513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:39.570085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.570390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.570699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] PATH_INFO: `/v3/users/30663d726cb242cf990bca67ea4649ac` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.575875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=30663d726cb242cf990bca67ea4649ac)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:39.580160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015699a5-2c69-4522-99a1-6eb4274c429e service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:39.581911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:39.598080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493145c-0452-4cf2-b587-b416c659d6f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:39.598746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493145c-0452-4cf2-b587-b416c659d6f2 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.599003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493145c-0452-4cf2-b587-b416c659d6f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.599271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493145c-0452-4cf2-b587-b416c659d6f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.608620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493145c-0452-4cf2-b587-b416c659d6f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:39.610351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493145c-0452-4cf2-b587-b416c659d6f2 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:39.638172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 845/1705] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:39.639067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493145c-0452-4cf2-b587-b416c659d6f2 service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:39.663668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493145c-0452-4cf2-b587-b416c659d6f2 service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:39.675108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca519f2-debf-4df1-ab93-928a12ee7b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:39.675731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca519f2-debf-4df1-ab93-928a12ee7b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.676011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca519f2-debf-4df1-ab93-928a12ee7b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.676293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca519f2-debf-4df1-ab93-928a12ee7b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.676930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca519f2-debf-4df1-ab93-928a12ee7b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:39.678574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca519f2-debf-4df1-ab93-928a12ee7b31 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:39.698242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30663d726cb242cf990bca67ea4649ac'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:39.702858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 846/1706] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:39.707430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca519f2-debf-4df1-ab93-928a12ee7b31 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:39.723479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30663d726cb242cf990bca67ea4649ac'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:39.724157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30663d726cb242cf990bca67ea4649ac'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:39.736046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca519f2-debf-4df1-ab93-928a12ee7b31 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:39.752346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:39.752916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30663d726cb242cf990bca67ea4649ac'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:39.753102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.753326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.753601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.754249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:39.754694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f803b43-4da8-45a0-b711-fdc54eef1cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30663d726cb242cf990bca67ea4649ac'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:39.755839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:39.784055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 861/1707] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:39 2022] DELETE /identity/v3/users/30663d726cb242cf990bca67ea4649ac => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:39.788138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5835475-7184-4cd1-80d8-75e35afa4037 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:39.788411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5835475-7184-4cd1-80d8-75e35afa4037 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:39.788749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5835475-7184-4cd1-80d8-75e35afa4037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:39.791475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 847/1708] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:39.792498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:39.797708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:39.832547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f5835475-7184-4cd1-80d8-75e35afa4037 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:39.833854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f5835475-7184-4cd1-80d8-75e35afa4037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:39.845602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6705-c69e-4682-adfa-1a050a445a33 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:39.989770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 862/1709] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:40.045452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5835475-7184-4cd1-80d8-75e35afa4037 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:40.046926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f5835475-7184-4cd1-80d8-75e35afa4037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:40.055105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94120d6d-6fff-4200-80e9-a3a3f1b61757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:40.062785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94120d6d-6fff-4200-80e9-a3a3f1b61757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:40.063259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94120d6d-6fff-4200-80e9-a3a3f1b61757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:40.065028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94120d6d-6fff-4200-80e9-a3a3f1b61757 admin admin] PATH_INFO: `/v3/projects/d44a6bc2060c41eaacf8fa27aaf0ef5d/users/109e3debf81b49c5be41474ee7b10aa6/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:40.096315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94120d6d-6fff-4200-80e9-a3a3f1b61757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44a6bc2060c41eaacf8fa27aaf0ef5d, user_id=109e3debf81b49c5be41474ee7b10aa6, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:40.104035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94120d6d-6fff-4200-80e9-a3a3f1b61757 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:40.138877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:40.155887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 848/1710] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:39 2022] PUT /identity/v3/projects/d44a6bc2060c41eaacf8fa27aaf0ef5d/users/109e3debf81b49c5be41474ee7b10aa6/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:40.162397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5835475-7184-4cd1-80d8-75e35afa4037 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:40.168054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dc784-9b76-49c9-a57d-924f4ccce060 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:40.168338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dc784-9b76-49c9-a57d-924f4ccce060 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:40.168612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dc784-9b76-49c9-a57d-924f4ccce060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:40.220692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-727dc784-9b76-49c9-a57d-924f4ccce060 None None] MFA Rules not processed for user `109e3debf81b49c5be41474ee7b10aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:40.228211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-727dc784-9b76-49c9-a57d-924f4ccce060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:40.272591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:40.297674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:40.298437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:40.299845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:40.301294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:40.453101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 849/1711] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:40.458723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dc784-9b76-49c9-a57d-924f4ccce060 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:40.504654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:40.507548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:40.533428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 863/1712] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:40.535125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:40.536825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:40.556963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7a81-4b4e-4896-8610-7d04adb1032d service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:40.576747 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:40.577484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:40.577902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:40.578348 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] PATH_INFO: `/v3/users/08d8e5ff2a9f448abfce6faa3e20162c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:40.581161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d8e5ff2a9f448abfce6faa3e20162c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:40.585051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:40.585577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dc784-9b76-49c9-a57d-924f4ccce060 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:40.596113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cd183-4660-49a2-8bfe-7b0f0b44d217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:40.596760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cd183-4660-49a2-8bfe-7b0f0b44d217 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:40.597044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cd183-4660-49a2-8bfe-7b0f0b44d217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:40.597343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cd183-4660-49a2-8bfe-7b0f0b44d217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:40.598500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cd183-4660-49a2-8bfe-7b0f0b44d217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:40.603277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cd183-4660-49a2-8bfe-7b0f0b44d217 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:40.652259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 850/1713] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:40 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:40.656101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cd183-4660-49a2-8bfe-7b0f0b44d217 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:40.689735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d8e5ff2a9f448abfce6faa3e20162c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:40.703539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d8e5ff2a9f448abfce6faa3e20162c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:40.704174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d8e5ff2a9f448abfce6faa3e20162c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:40.731847 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d8e5ff2a9f448abfce6faa3e20162c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:40.733075 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33ead864-2456-423b-9ed5-96dbe87dfbe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d8e5ff2a9f448abfce6faa3e20162c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:40.757927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 864/1714] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:40 2022] DELETE /identity/v3/users/08d8e5ff2a9f448abfce6faa3e20162c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:40.763020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1861e2c-8b43-43d6-9d9d-9cae36ae7035 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:40.763832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1861e2c-8b43-43d6-9d9d-9cae36ae7035 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:40.765765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1861e2c-8b43-43d6-9d9d-9cae36ae7035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:40.808428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cd183-4660-49a2-8bfe-7b0f0b44d217 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:40.813618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b1861e2c-8b43-43d6-9d9d-9cae36ae7035 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:40.818905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b1861e2c-8b43-43d6-9d9d-9cae36ae7035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:40.860488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:40.910865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:40.930796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:40.933443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:40.934729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:40.935223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:40.937359 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:40.951180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 865/1715] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:40.955773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:41.004423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 851/1716] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:40 2022] POST /identity/v3/users => generated 297 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:41.009385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1861e2c-8b43-43d6-9d9d-9cae36ae7035 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:41.010903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b1861e2c-8b43-43d6-9d9d-9cae36ae7035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:41.033336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1861e2c-8b43-43d6-9d9d-9cae36ae7035 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:41.054350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71844a4-7d6c-40c0-b35e-03f1b908c3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:41.056972 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71844a4-7d6c-40c0-b35e-03f1b908c3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:41.057287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:41.057774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71844a4-7d6c-40c0-b35e-03f1b908c3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:41.059115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71844a4-7d6c-40c0-b35e-03f1b908c3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:41.060094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71844a4-7d6c-40c0-b35e-03f1b908c3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:41.063012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71844a4-7d6c-40c0-b35e-03f1b908c3e7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:41.068167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:41.089295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 866/1717] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:41.094353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71844a4-7d6c-40c0-b35e-03f1b908c3e7 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:41.133169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71844a4-7d6c-40c0-b35e-03f1b908c3e7 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:41.152183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d055-c55e-4d34-a46d-155148453a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:41.153464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d055-c55e-4d34-a46d-155148453a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:41.155584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d055-c55e-4d34-a46d-155148453a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:41.156286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d055-c55e-4d34-a46d-155148453a24 admin admin] PATH_INFO: `/v3/projects/fc0b9f9d151840448b9e440429ef776b/users/233196973602453bad2efb14ec905ded/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:41.197479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75d055-c55e-4d34-a46d-155148453a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0b9f9d151840448b9e440429ef776b, user_id=233196973602453bad2efb14ec905ded, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:41.202439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75d055-c55e-4d34-a46d-155148453a24 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:41.226690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f793f9-9435-4a14-8026-0e2a535ebc81 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:41.238969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 867/1718] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:41 2022] PUT /identity/v3/projects/fc0b9f9d151840448b9e440429ef776b/users/233196973602453bad2efb14ec905ded/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:41.240603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d055-c55e-4d34-a46d-155148453a24 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:41.400567 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:41.404050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:41.405653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:41.405891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:41.455778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d055-c55e-4d34-a46d-155148453a24 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:41.475427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318efa8-41ad-4392-b704-e2c08a07c7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:41.478862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318efa8-41ad-4392-b704-e2c08a07c7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:41.479296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318efa8-41ad-4392-b704-e2c08a07c7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:41.479544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318efa8-41ad-4392-b704-e2c08a07c7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:41.482527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318efa8-41ad-4392-b704-e2c08a07c7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:41.490303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318efa8-41ad-4392-b704-e2c08a07c7b3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:41.522220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 868/1719] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:37:41.524397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318efa8-41ad-4392-b704-e2c08a07c7b3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:41.550056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318efa8-41ad-4392-b704-e2c08a07c7b3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:41.562665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aef8f4-b079-4a86-a426-e5abe23cffe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:41.562857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef8f4-b079-4a86-a426-e5abe23cffe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:41.563018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef8f4-b079-4a86-a426-e5abe23cffe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:41.563296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef8f4-b079-4a86-a426-e5abe23cffe4 admin admin] PATH_INFO: `/v3/projects/fc0b9f9d151840448b9e440429ef776b/users/233196973602453bad2efb14ec905ded/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:41.571158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aef8f4-b079-4a86-a426-e5abe23cffe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0b9f9d151840448b9e440429ef776b, user_id=233196973602453bad2efb14ec905ded, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:41.578963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aef8f4-b079-4a86-a426-e5abe23cffe4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:41.617319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 869/1720] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:41 2022] PUT /identity/v3/projects/fc0b9f9d151840448b9e440429ef776b/users/233196973602453bad2efb14ec905ded/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:41.628272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6402d-d5bc-48c4-b37c-e2f9695fe5b7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:41.628646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:41.629067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6402d-d5bc-48c4-b37c-e2f9695fe5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:41.629067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6402d-d5bc-48c4-b37c-e2f9695fe5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:41.630156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:41.653955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 852/1721] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:41.706683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-53b6402d-d5bc-48c4-b37c-e2f9695fe5b7 None None] MFA Rules not processed for user `233196973602453bad2efb14ec905ded`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:41.708239 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-53b6402d-d5bc-48c4-b37c-e2f9695fe5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:41.801566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:41.815117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 870/1722] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:41.827947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:41.830693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:41.926280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:41.946886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9902-e628-4b39-8db8-c797449397dc service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:42.081532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:42.082172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:42.082454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:42.084241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:42.114154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:42.115631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:42.155517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 853/1723] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:42.299897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6402d-d5bc-48c4-b37c-e2f9695fe5b7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:42.318559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6402d-d5bc-48c4-b37c-e2f9695fe5b7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:42.330227 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:42.331114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:42.331538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:42.331942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:42.342393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:42.482627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:42.484560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:42.514215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 871/1724] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:42.956294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:42.957954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:43.152277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec0a1-6683-4199-9ae7-478c3b18c569 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:43.167729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:43.168689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:43.169292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:43.169535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] PATH_INFO: `/v3/projects/dcabbb9bbd954ff38b77a6c30f5f2268` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:43.171088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcabbb9bbd954ff38b77a6c30f5f2268)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:43.174270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:43.285105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcabbb9bbd954ff38b77a6c30f5f2268', 'request_id': 'req-934795bd-f17c-4153-82dc-7e2d95a9ef9e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.295601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcabbb9bbd954ff38b77a6c30f5f2268', 'request_id': 'req-934795bd-f17c-4153-82dc-7e2d95a9ef9e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.314610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcabbb9bbd954ff38b77a6c30f5f2268', 'request_id': 'req-934795bd-f17c-4153-82dc-7e2d95a9ef9e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.341646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] The token cache is being invalidate because project dcabbb9bbd954ff38b77a6c30f5f2268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:43.341916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.344129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.348341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 854/1725] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:42 2022] DELETE /identity/v3/projects/dcabbb9bbd954ff38b77a6c30f5f2268 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:43.352874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:43.369308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f4a7d-d7e3-4f52-a5e3-f62720c1bd41 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:43.470862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:43.471976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:43.472475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:43.472958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] PATH_INFO: `/v3/users/301953bab3344be79804e96644ef460f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:43.475540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=301953bab3344be79804e96644ef460f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:43.481112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:43.579520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301953bab3344be79804e96644ef460f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.600252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301953bab3344be79804e96644ef460f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.600960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301953bab3344be79804e96644ef460f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.624398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301953bab3344be79804e96644ef460f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.627209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e556276-ce22-499e-a8ac-4042d830d170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301953bab3344be79804e96644ef460f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:43.652908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 872/1726] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:43 2022] DELETE /identity/v3/users/301953bab3344be79804e96644ef460f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:43.958368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:43.960006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:43.983652 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934795bd-f17c-4153-82dc-7e2d95a9ef9e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:44.121737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5974f3-d92b-4f0d-9c18-2458eb9c729f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:44.122017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5974f3-d92b-4f0d-9c18-2458eb9c729f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:44.122287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5974f3-d92b-4f0d-9c18-2458eb9c729f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:44.125606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:44.239033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:44.241188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:44.241632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:44.242199 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:44.242846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] PATH_INFO: `/v3/projects/1bc3596983554ce993c08879363c83b1` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:44.245824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc3596983554ce993c08879363c83b1)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:44.249532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:44.256529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8e5974f3-d92b-4f0d-9c18-2458eb9c729f None None] MFA Rules not processed for user `a3d9e5dc719c430eac6b5c8d8b814566`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:44.258053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8e5974f3-d92b-4f0d-9c18-2458eb9c729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:44.446971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc3596983554ce993c08879363c83b1', 'request_id': 'req-d0d5fc07-9857-4d74-81de-e34c57cb0d86'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:44.458049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc3596983554ce993c08879363c83b1', 'request_id': 'req-d0d5fc07-9857-4d74-81de-e34c57cb0d86'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:44.478687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc3596983554ce993c08879363c83b1', 'request_id': 'req-d0d5fc07-9857-4d74-81de-e34c57cb0d86'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:44.511201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] The token cache is being invalidate because project 1bc3596983554ce993c08879363c83b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:44.512076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:44.512375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 873/1727] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:44.531625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:44.541305 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5974f3-d92b-4f0d-9c18-2458eb9c729f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:44.543081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8e5974f3-d92b-4f0d-9c18-2458eb9c729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:44.543976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 855/1728] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:43 2022] DELETE /identity/v3/projects/1bc3596983554ce993c08879363c83b1 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:44.551062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:44.557537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:44.744196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5974f3-d92b-4f0d-9c18-2458eb9c729f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:44.801844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5fc07-9857-4d74-81de-e34c57cb0d86 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:44.830415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:44.831352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:44.831779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:44.832248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] PATH_INFO: `/v3/users/fd8e494e5edc4c72bc71e76fd783479f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:44.839462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8e494e5edc4c72bc71e76fd783479f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:44.844482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:44.953687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dea1fd-8211-42f0-8f1d-f9171b18e38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:44.954751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dea1fd-8211-42f0-8f1d-f9171b18e38f service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:44.955031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dea1fd-8211-42f0-8f1d-f9171b18e38f service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:44.955597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dea1fd-8211-42f0-8f1d-f9171b18e38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:44.965584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f3dea1fd-8211-42f0-8f1d-f9171b18e38f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:45.009791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8e494e5edc4c72bc71e76fd783479f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:45.028735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8e494e5edc4c72bc71e76fd783479f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:45.030360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8e494e5edc4c72bc71e76fd783479f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:45.053018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8e494e5edc4c72bc71e76fd783479f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:45.059403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:45.078862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8e494e5edc4c72bc71e76fd783479f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:45.105662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 856/1729] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:44 2022] DELETE /identity/v3/users/fd8e494e5edc4c72bc71e76fd783479f => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:45.121588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:45.230498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dea1fd-8211-42f0-8f1d-f9171b18e38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:45.232144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dea1fd-8211-42f0-8f1d-f9171b18e38f service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:45.292288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 874/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 752 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:45.492169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:45.494807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:45.757703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee693d3-60a0-42ac-8180-e60fd8b75fb8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:45.791750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:45.792573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:45.792682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:45.792954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] PATH_INFO: `/v3/projects/9e929e12019045b682178da8a644a0f4` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:45.799247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e929e12019045b682178da8a644a0f4)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:45.802258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:45.939973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e929e12019045b682178da8a644a0f4', 'request_id': 'req-d87848eb-dff2-4b06-b519-ac26d53dec47'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:45.957441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e929e12019045b682178da8a644a0f4', 'request_id': 'req-d87848eb-dff2-4b06-b519-ac26d53dec47'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:45.981058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e929e12019045b682178da8a644a0f4', 'request_id': 'req-d87848eb-dff2-4b06-b519-ac26d53dec47'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:46.023511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] The token cache is being invalidate because project 9e929e12019045b682178da8a644a0f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:46.023511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:46.024412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d87848eb-dff2-4b06-b519-ac26d53dec47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:46.026633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 857/1731] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:45 2022] DELETE /identity/v3/projects/9e929e12019045b682178da8a644a0f4 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:46.032350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1c6ec-967c-4e49-99c6-3b9b0dea6007 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:46.032884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1c6ec-967c-4e49-99c6-3b9b0dea6007 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:46.032884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1c6ec-967c-4e49-99c6-3b9b0dea6007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:46.074903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-80b1c6ec-967c-4e49-99c6-3b9b0dea6007 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:46.080133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-80b1c6ec-967c-4e49-99c6-3b9b0dea6007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:46.264797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 875/1732] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:46.278537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c65c53-cc9b-433d-91ed-13a3a97aba99 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:46.278986 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c65c53-cc9b-433d-91ed-13a3a97aba99 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:46.279400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c65c53-cc9b-433d-91ed-13a3a97aba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:46.319536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-19c65c53-cc9b-433d-91ed-13a3a97aba99 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:46.322970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-19c65c53-cc9b-433d-91ed-13a3a97aba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:46.347701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 858/1733] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:46.352761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1c6ec-967c-4e49-99c6-3b9b0dea6007 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:46.369618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1c6ec-967c-4e49-99c6-3b9b0dea6007 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:46.379544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:46.380579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:46.381024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:46.381533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:46.382437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:46.383999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:46.386933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:46.423280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c65c53-cc9b-433d-91ed-13a3a97aba99 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:46.433918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c65c53-cc9b-433d-91ed-13a3a97aba99 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:46.434592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 876/1734] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:46.450613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6999db-c83e-43a4-9d97-bee9f38db9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:46.451702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6999db-c83e-43a4-9d97-bee9f38db9e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:46.452132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6999db-c83e-43a4-9d97-bee9f38db9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:46.452549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6999db-c83e-43a4-9d97-bee9f38db9e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:46.453577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6999db-c83e-43a4-9d97-bee9f38db9e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:46.457939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6999db-c83e-43a4-9d97-bee9f38db9e5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:46.518743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 859/1735] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:46 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:46.523085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:46.543055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22457a1e-f381-4bae-b8b9-8091d497c5b4 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:46.713213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381f98e-0502-4351-a53d-5dae3ede4341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:46.713683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381f98e-0502-4351-a53d-5dae3ede4341 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:46.713997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381f98e-0502-4351-a53d-5dae3ede4341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:46.714486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381f98e-0502-4351-a53d-5dae3ede4341 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:46.716390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381f98e-0502-4351-a53d-5dae3ede4341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:46.722410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381f98e-0502-4351-a53d-5dae3ede4341 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:46.776329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 877/1736] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:46 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:46.777921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6999db-c83e-43a4-9d97-bee9f38db9e5 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:46.793385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6999db-c83e-43a4-9d97-bee9f38db9e5 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:46.807563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1928-ee73-440c-847c-755f185420bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:46.808216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1928-ee73-440c-847c-755f185420bf admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:46.808495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1928-ee73-440c-847c-755f185420bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:46.808765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1928-ee73-440c-847c-755f185420bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:46.809470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b1928-ee73-440c-847c-755f185420bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:46.820849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b1928-ee73-440c-847c-755f185420bf admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:46.847128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 860/1737] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:46.852800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381f98e-0502-4351-a53d-5dae3ede4341 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:46.869341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381f98e-0502-4351-a53d-5dae3ede4341 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:46.881965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0ee4d-44a9-474c-bb38-e4167b1340f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:46.882692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ee4d-44a9-474c-bb38-e4167b1340f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:46.883103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ee4d-44a9-474c-bb38-e4167b1340f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:46.883494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ee4d-44a9-474c-bb38-e4167b1340f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:46.884241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0ee4d-44a9-474c-bb38-e4167b1340f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:46.892428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0ee4d-44a9-474c-bb38-e4167b1340f8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:46.921369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 878/1738] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:46.923216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1928-ee73-440c-847c-755f185420bf admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:46.937102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1928-ee73-440c-847c-755f185420bf admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:46.950573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:46.951102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:46.951452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:46.951847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] PATH_INFO: `/v3/projects/160b012805e34626b213f728100f7add/users/2cd906b5175e4ec5b99accba68b4d0da/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:46.996635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=160b012805e34626b213f728100f7add, user_id=2cd906b5175e4ec5b99accba68b4d0da, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:46.996635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:47.013284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 861/1739] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:46 2022] PUT /identity/v3/projects/160b012805e34626b213f728100f7add/users/2cd906b5175e4ec5b99accba68b4d0da/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:47.025685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3560cfa-18f5-45ec-9528-db15204d1f91 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:47.026035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3560cfa-18f5-45ec-9528-db15204d1f91 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:47.026394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3560cfa-18f5-45ec-9528-db15204d1f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:47.095155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b3560cfa-18f5-45ec-9528-db15204d1f91 None None] MFA Rules not processed for user `2cd906b5175e4ec5b99accba68b4d0da`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:47.098635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b3560cfa-18f5-45ec-9528-db15204d1f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:47.186216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:47.259639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 879/1740] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:47.278849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:47.280880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:47.303934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:47.454232 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587a222-9bfc-4ddd-8697-03e48b1e19ff admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:47.467668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:47.468635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:47.469146 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:47.469696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:47.506610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:47.510516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:47.541687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 862/1741] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:47.585275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3560cfa-18f5-45ec-9528-db15204d1f91 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:47.762571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3560cfa-18f5-45ec-9528-db15204d1f91 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:47.786241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:47.787012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:47.787284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:47.787559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] PATH_INFO: `/v3/users/2cd906b5175e4ec5b99accba68b4d0da` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:47.790313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd906b5175e4ec5b99accba68b4d0da)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:47.792335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:47.979227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:47.990197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:48.013793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd906b5175e4ec5b99accba68b4d0da'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.028775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730567f0-ac2d-4c77-a558-7fe7553d67c3 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:48.036409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd906b5175e4ec5b99accba68b4d0da'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.037420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd906b5175e4ec5b99accba68b4d0da'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.061704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:48.062658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:48.063109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:48.063581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] PATH_INFO: `/v3/projects/2b70196cc87c4d3a9b6d8601225aa63e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:48.070456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b70196cc87c4d3a9b6d8601225aa63e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:48.079078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:48.102698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd906b5175e4ec5b99accba68b4d0da'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.104846 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b2517f33-2cb1-40a2-8b0e-c9fbf9a8df2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd906b5175e4ec5b99accba68b4d0da'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.157416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 880/1742] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:47 2022] DELETE /identity/v3/users/2cd906b5175e4ec5b99accba68b4d0da => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:48.163651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b81fb0-5777-44ec-9a3b-6a5185361500 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:48.164369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b81fb0-5777-44ec-9a3b-6a5185361500 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:48.165049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b81fb0-5777-44ec-9a3b-6a5185361500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:48.283657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-49b81fb0-5777-44ec-9a3b-6a5185361500 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:48.287401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-49b81fb0-5777-44ec-9a3b-6a5185361500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:48.317272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b70196cc87c4d3a9b6d8601225aa63e', 'request_id': 'req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.350086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b70196cc87c4d3a9b6d8601225aa63e', 'request_id': 'req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.371701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b70196cc87c4d3a9b6d8601225aa63e', 'request_id': 'req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.382954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:48.431452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] The token cache is being invalidate because project 2b70196cc87c4d3a9b6d8601225aa63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:48.432236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.433803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:48.439495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 881/1744] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:48.441802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 863/1744] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:47 2022] DELETE /identity/v3/projects/2b70196cc87c4d3a9b6d8601225aa63e => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:48.443618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6d41-9743-4235-be2b-b4752157ecfd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:48.444334 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6d41-9743-4235-be2b-b4752157ecfd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:48.444822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6d41-9743-4235-be2b-b4752157ecfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:48.476248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-284e6d41-9743-4235-be2b-b4752157ecfd None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:48.478951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-284e6d41-9743-4235-be2b-b4752157ecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:48.491570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:48.493139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:48.498470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 882/1745] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:48.511039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb20afb-68a0-4bf3-ba3a-dc28f2d67664 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:48.517209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb20afb-68a0-4bf3-ba3a-dc28f2d67664 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:48.517728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb20afb-68a0-4bf3-ba3a-dc28f2d67664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:48.578469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-efb20afb-68a0-4bf3-ba3a-dc28f2d67664 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:48.580260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-efb20afb-68a0-4bf3-ba3a-dc28f2d67664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:48.617522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 883/1746] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:48.624357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb20afb-68a0-4bf3-ba3a-dc28f2d67664 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:48.642801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb20afb-68a0-4bf3-ba3a-dc28f2d67664 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:48.657979 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:48.658603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:48.658881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:48.659148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:48.660596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:48.660921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:48.667403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:48.669603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ceaf20-8e17-4c07-887c-f9f91ee6f58f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:48.674151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c828-0645-42dd-8553-222072fda7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:48.675091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c828-0645-42dd-8553-222072fda7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:48.675585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c828-0645-42dd-8553-222072fda7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:48.676119 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c828-0645-42dd-8553-222072fda7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:48.679852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8b76c828-0645-42dd-8553-222072fda7d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:48.693262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 884/1747] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:48.698328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:48.702329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c828-0645-42dd-8553-222072fda7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:48.704752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c828-0645-42dd-8553-222072fda7d1 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:48.725733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a11cb9-699e-4f30-947f-c384cedf6df1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:48.734144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 864/1748] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:48.742006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fde8e-3c74-4151-8b31-b2c86234f760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:48.743316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fde8e-3c74-4151-8b31-b2c86234f760 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:48.743593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fde8e-3c74-4151-8b31-b2c86234f760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:48.743866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fde8e-3c74-4151-8b31-b2c86234f760 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:48.744504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fde8e-3c74-4151-8b31-b2c86234f760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:48.754279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fde8e-3c74-4151-8b31-b2c86234f760 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:48.768521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60f4f-8f75-4d10-9212-d24ca3e283de None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:48.769133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60f4f-8f75-4d10-9212-d24ca3e283de None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:48.769664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60f4f-8f75-4d10-9212-d24ca3e283de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:48.821324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 885/1749] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:48 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:48.825941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fde8e-3c74-4151-8b31-b2c86234f760 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:48.841216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-93b60f4f-8f75-4d10-9212-d24ca3e283de None None] MFA Rules not processed for user `258ccb4bb15a4747b4c17d2a26645888`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:48.844404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-93b60f4f-8f75-4d10-9212-d24ca3e283de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:48.961680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:49.024096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fde8e-3c74-4151-8b31-b2c86234f760 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:49.038036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb30666-1dae-40e9-bf57-6f70660558f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:49.038675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30666-1dae-40e9-bf57-6f70660558f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.038953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30666-1dae-40e9-bf57-6f70660558f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.039220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30666-1dae-40e9-bf57-6f70660558f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.040412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb30666-1dae-40e9-bf57-6f70660558f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:49.042408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb30666-1dae-40e9-bf57-6f70660558f6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:49.105688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 865/1750] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:49.107933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 886/1751] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:48 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:49.109272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60f4f-8f75-4d10-9212-d24ca3e283de None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:49.118378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefaa99-e055-45df-a3ed-ef1004b10872 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.118901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefaa99-e055-45df-a3ed-ef1004b10872 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.119326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefaa99-e055-45df-a3ed-ef1004b10872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.139027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60f4f-8f75-4d10-9212-d24ca3e283de None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:49.151893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768593d8-a8e9-4bf0-910c-11ed2907dabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:49.153994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ddefaa99-e055-45df-a3ed-ef1004b10872 None None] MFA Rules not processed for user `258ccb4bb15a4747b4c17d2a26645888`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:49.160494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768593d8-a8e9-4bf0-910c-11ed2907dabb admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.160942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768593d8-a8e9-4bf0-910c-11ed2907dabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.162000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768593d8-a8e9-4bf0-910c-11ed2907dabb admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.162796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768593d8-a8e9-4bf0-910c-11ed2907dabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:49.165870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ddefaa99-e055-45df-a3ed-ef1004b10872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:49.169760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768593d8-a8e9-4bf0-910c-11ed2907dabb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:49.183465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 887/1752] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:49.190566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefaa99-e055-45df-a3ed-ef1004b10872 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:49.192014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ddefaa99-e055-45df-a3ed-ef1004b10872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:49.198733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 866/1753] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:49.202505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768593d8-a8e9-4bf0-910c-11ed2907dabb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:49.221490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768593d8-a8e9-4bf0-910c-11ed2907dabb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:49.241202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbed6b6-bfed-4305-b67e-cade5e0abdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:49.242122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbed6b6-bfed-4305-b67e-cade5e0abdbb admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.242802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbed6b6-bfed-4305-b67e-cade5e0abdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.243511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbed6b6-bfed-4305-b67e-cade5e0abdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.244355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbed6b6-bfed-4305-b67e-cade5e0abdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:49.254418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbed6b6-bfed-4305-b67e-cade5e0abdbb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:49.289124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 867/1754] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:49.293594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbed6b6-bfed-4305-b67e-cade5e0abdbb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:49.307268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbed6b6-bfed-4305-b67e-cade5e0abdbb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:49.313146 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2fc08-bfdc-4f4e-908f-e525e1c0497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:49.314202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fc08-bfdc-4f4e-908f-e525e1c0497b admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.314959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fc08-bfdc-4f4e-908f-e525e1c0497b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.315445 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fc08-bfdc-4f4e-908f-e525e1c0497b admin admin] PATH_INFO: `/v3/projects/9779bfaefa1140bab7db560eb45f0019/users/bec2211b18a0499db1caeba46670a9d5/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.346300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2fc08-bfdc-4f4e-908f-e525e1c0497b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9779bfaefa1140bab7db560eb45f0019, user_id=bec2211b18a0499db1caeba46670a9d5, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:49.350393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2fc08-bfdc-4f4e-908f-e525e1c0497b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:49.372469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 868/1755] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:49 2022] PUT /identity/v3/projects/9779bfaefa1140bab7db560eb45f0019/users/bec2211b18a0499db1caeba46670a9d5/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:37:49.387869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b9a0-7dd3-48cf-8892-17e50ed53ac1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.391202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b9a0-7dd3-48cf-8892-17e50ed53ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.391580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b9a0-7dd3-48cf-8892-17e50ed53ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.402815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefaa99-e055-45df-a3ed-ef1004b10872 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:49.441280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:49.501176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-68c5b9a0-7dd3-48cf-8892-17e50ed53ac1 None None] MFA Rules not processed for user `bec2211b18a0499db1caeba46670a9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:49.505950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-68c5b9a0-7dd3-48cf-8892-17e50ed53ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:49.555615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ee7f6-de36-4400-b5c7-575c194bcea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:49.556369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ee7f6-de36-4400-b5c7-575c194bcea9 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.556787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ee7f6-de36-4400-b5c7-575c194bcea9 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.557119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ee7f6-de36-4400-b5c7-575c194bcea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.667808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ee7f6-de36-4400-b5c7-575c194bcea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:49.670389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ee7f6-de36-4400-b5c7-575c194bcea9 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:49.678539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 869/1756] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:49.684397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5b9a0-7dd3-48cf-8892-17e50ed53ac1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:49.690264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 888/1757] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:49.776397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5b9a0-7dd3-48cf-8892-17e50ed53ac1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:49.783743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a1e08-15d0-457d-9d08-b2d33c48be72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:49.784385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1e08-15d0-457d-9d08-b2d33c48be72 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.784667 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1e08-15d0-457d-9d08-b2d33c48be72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.784943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1e08-15d0-457d-9d08-b2d33c48be72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.785619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a1e08-15d0-457d-9d08-b2d33c48be72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:49.787883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a1e08-15d0-457d-9d08-b2d33c48be72 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:49.824636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:49.846461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 870/1758] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:49 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:49.847668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ee7f6-de36-4400-b5c7-575c194bcea9 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:49.864207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ee7f6-de36-4400-b5c7-575c194bcea9 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:49.973943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dac88-d203-4d28-9583-47e2de03dd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:49.974625 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dac88-d203-4d28-9583-47e2de03dd9d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:49.976296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dac88-d203-4d28-9583-47e2de03dd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:49.976638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dac88-d203-4d28-9583-47e2de03dd9d admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:49.977990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8dac88-d203-4d28-9583-47e2de03dd9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:49.980303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8dac88-d203-4d28-9583-47e2de03dd9d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:50.024703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 889/1759] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:49 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:50.028007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a1e08-15d0-457d-9d08-b2d33c48be72 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:50.047814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a1e08-15d0-457d-9d08-b2d33c48be72 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:50.054322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2b367-893e-4956-b6ac-06cb88ef83d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:50.055582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2b367-893e-4956-b6ac-06cb88ef83d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:50.056167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2b367-893e-4956-b6ac-06cb88ef83d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:50.056448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2b367-893e-4956-b6ac-06cb88ef83d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:50.057462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2b367-893e-4956-b6ac-06cb88ef83d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:50.059431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2b367-893e-4956-b6ac-06cb88ef83d8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:50.103896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 871/1760] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:37:50.108450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dac88-d203-4d28-9583-47e2de03dd9d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:50.133098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dac88-d203-4d28-9583-47e2de03dd9d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:50.145822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f215087-c24a-40de-ae50-4fc9b9744afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:50.147021 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f215087-c24a-40de-ae50-4fc9b9744afd admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:50.147308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f215087-c24a-40de-ae50-4fc9b9744afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:50.147695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f215087-c24a-40de-ae50-4fc9b9744afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:50.149704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f215087-c24a-40de-ae50-4fc9b9744afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:50.158562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f215087-c24a-40de-ae50-4fc9b9744afd admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:50.194436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 890/1761] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:50.198022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2b367-893e-4956-b6ac-06cb88ef83d8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:50.217544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2b367-893e-4956-b6ac-06cb88ef83d8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:50.235681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6eb01-bc5a-4be3-a4a5-78586e67edfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:50.236392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6eb01-bc5a-4be3-a4a5-78586e67edfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:50.236664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6eb01-bc5a-4be3-a4a5-78586e67edfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:50.236942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6eb01-bc5a-4be3-a4a5-78586e67edfe admin admin] PATH_INFO: `/v3/projects/970e12eb04c042018f72bc5d8b8abc55/users/9e49bf6589144816add2e325c2cf9fd7/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:50.294279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6eb01-bc5a-4be3-a4a5-78586e67edfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=970e12eb04c042018f72bc5d8b8abc55, user_id=9e49bf6589144816add2e325c2cf9fd7, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:50.302446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6eb01-bc5a-4be3-a4a5-78586e67edfe admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:50.329747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 872/1762] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:50 2022] PUT /identity/v3/projects/970e12eb04c042018f72bc5d8b8abc55/users/9e49bf6589144816add2e325c2cf9fd7/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:50.343378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac407-1014-4c0d-b996-d907a604f236 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:50.343682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac407-1014-4c0d-b996-d907a604f236 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:50.343949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac407-1014-4c0d-b996-d907a604f236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:50.455194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b58ac407-1014-4c0d-b996-d907a604f236 None None] MFA Rules not processed for user `9e49bf6589144816add2e325c2cf9fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:50.463403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b58ac407-1014-4c0d-b996-d907a604f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:50.566403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:50.649612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 891/1763] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:50.654214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6eb01-bc5a-4be3-a4a5-78586e67edfe admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:50.801269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6eb01-bc5a-4be3-a4a5-78586e67edfe admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:50.808442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b0b0d-2024-4be7-ac7d-3180e1fec879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:50.809900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b0b0d-2024-4be7-ac7d-3180e1fec879 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:50.810511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b0b0d-2024-4be7-ac7d-3180e1fec879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:50.811043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b0b0d-2024-4be7-ac7d-3180e1fec879 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:50.812809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b0b0d-2024-4be7-ac7d-3180e1fec879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:50.818604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b0b0d-2024-4be7-ac7d-3180e1fec879 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:50.877649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ac407-1014-4c0d-b996-d907a604f236 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:50.879850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 873/1764] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:50 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:50.900598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ac407-1014-4c0d-b996-d907a604f236 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:50.997693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e92dc0-b4e5-4f9c-960a-1cbf32754337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:50.999801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92dc0-b4e5-4f9c-960a-1cbf32754337 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.000108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92dc0-b4e5-4f9c-960a-1cbf32754337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.000385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92dc0-b4e5-4f9c-960a-1cbf32754337 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.001101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e92dc0-b4e5-4f9c-960a-1cbf32754337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:51.005466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e92dc0-b4e5-4f9c-960a-1cbf32754337 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:51.037268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7043d8-dac8-4a36-a791-cf30cd87c08c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.037968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7043d8-dac8-4a36-a791-cf30cd87c08c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.038339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7043d8-dac8-4a36-a791-cf30cd87c08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.073751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 892/1765] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:50 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:51.077306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e92dc0-b4e5-4f9c-960a-1cbf32754337 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:51.083633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6d7043d8-dac8-4a36-a791-cf30cd87c08c None None] MFA Rules not processed for user `a3d9e5dc719c430eac6b5c8d8b814566`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:51.086143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6d7043d8-dac8-4a36-a791-cf30cd87c08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:51.098242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e92dc0-b4e5-4f9c-960a-1cbf32754337 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:51.111914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853e52b-6ce9-4624-8d1d-18ee1e839f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:51.112865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853e52b-6ce9-4624-8d1d-18ee1e839f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.113062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853e52b-6ce9-4624-8d1d-18ee1e839f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.113675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853e52b-6ce9-4624-8d1d-18ee1e839f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.114013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853e52b-6ce9-4624-8d1d-18ee1e839f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:51.115899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853e52b-6ce9-4624-8d1d-18ee1e839f9a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:51.132906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 893/1766] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:37:51.134834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac6c7-22e0-421a-a014-9dbc7c8e3a7f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.135133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac6c7-22e0-421a-a014-9dbc7c8e3a7f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.135383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac6c7-22e0-421a-a014-9dbc7c8e3a7f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.149683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 894/1767] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 10 13:37:51 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:37:51.150806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ac6c7-22e0-421a-a014-9dbc7c8e3a7f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:51.176960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ac6c7-22e0-421a-a014-9dbc7c8e3a7f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:51.182313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a907ea-a566-4d5d-bbe5-591fb1182144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:51.183282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a907ea-a566-4d5d-bbe5-591fb1182144 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.183282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a907ea-a566-4d5d-bbe5-591fb1182144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.183956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a907ea-a566-4d5d-bbe5-591fb1182144 admin admin] PATH_INFO: `/v3/projects/499c252e7b614000a08aca9b2adfd27f/users/1f6d3c7ed5c94f579cba6c77d762641e/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.212754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 874/1768] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:51.215062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473fb43-fc4b-45b4-b9e3-a95ac9c31dfd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.215349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473fb43-fc4b-45b4-b9e3-a95ac9c31dfd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.215903 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473fb43-fc4b-45b4-b9e3-a95ac9c31dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.217690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a907ea-a566-4d5d-bbe5-591fb1182144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499c252e7b614000a08aca9b2adfd27f, user_id=1f6d3c7ed5c94f579cba6c77d762641e, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:51.220378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a907ea-a566-4d5d-bbe5-591fb1182144 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:51.254070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 895/1769] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:51 2022] PUT /identity/v3/projects/499c252e7b614000a08aca9b2adfd27f/users/1f6d3c7ed5c94f579cba6c77d762641e/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:51.257896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a907ea-a566-4d5d-bbe5-591fb1182144 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:51.287465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5473fb43-fc4b-45b4-b9e3-a95ac9c31dfd None None] MFA Rules not processed for user `234eed198ab84e95ae1a7f03785a1481`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:51.289090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5473fb43-fc4b-45b4-b9e3-a95ac9c31dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:51.357871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:51.402016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a907ea-a566-4d5d-bbe5-591fb1182144 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:51.415123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 875/1770] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:37:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:51.419895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b795b-c450-44a3-bc7c-a248a8b971a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:51.420794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b795b-c450-44a3-bc7c-a248a8b971a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.421382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b795b-c450-44a3-bc7c-a248a8b971a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.421885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b795b-c450-44a3-bc7c-a248a8b971a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.426297 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b795b-c450-44a3-bc7c-a248a8b971a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:51.428608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b795b-c450-44a3-bc7c-a248a8b971a1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:51.471570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 896/1771] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:37:51.473530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473fb43-fc4b-45b4-b9e3-a95ac9c31dfd None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:51.505507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473fb43-fc4b-45b4-b9e3-a95ac9c31dfd None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:51.520507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ef747-fdfa-4c57-901e-6c225fdbb9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:51.522031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ef747-fdfa-4c57-901e-6c225fdbb9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.522404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ef747-fdfa-4c57-901e-6c225fdbb9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.523044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ef747-fdfa-4c57-901e-6c225fdbb9d7 admin admin] PATH_INFO: `/v3/projects/499c252e7b614000a08aca9b2adfd27f/users/1f6d3c7ed5c94f579cba6c77d762641e/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.526783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ef747-fdfa-4c57-901e-6c225fdbb9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499c252e7b614000a08aca9b2adfd27f, user_id=1f6d3c7ed5c94f579cba6c77d762641e, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:51.530824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ef747-fdfa-4c57-901e-6c225fdbb9d7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:51.533219 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b795b-c450-44a3-bc7c-a248a8b971a1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:51.550433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b795b-c450-44a3-bc7c-a248a8b971a1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:51.560700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:51.561370 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.561902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.561902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.568250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 876/1772] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:51 2022] PUT /identity/v3/projects/499c252e7b614000a08aca9b2adfd27f/users/1f6d3c7ed5c94f579cba6c77d762641e/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:51.579848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e13c70-d203-4665-b46e-41b0c6ebdd22 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.581635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:51.582268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e13c70-d203-4665-b46e-41b0c6ebdd22 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.583117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:51.584795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e13c70-d203-4665-b46e-41b0c6ebdd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.604171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 897/1773] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:51.619652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:51.621109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:51.646980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2a0e-b8c1-4552-8ebb-271d694d3ddc service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:51.658585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:51.674370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-29e13c70-d203-4665-b46e-41b0c6ebdd22 None None] MFA Rules not processed for user `1f6d3c7ed5c94f579cba6c77d762641e`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:51.678189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-29e13c70-d203-4665-b46e-41b0c6ebdd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:51.779935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:51.780584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.780878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.781223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] PATH_INFO: `/v3/projects/160b012805e34626b213f728100f7add` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:51.783945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=160b012805e34626b213f728100f7add)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:51.785930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:51.886782 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 877/1774] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:51.898424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e13c70-d203-4665-b46e-41b0c6ebdd22 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:51.899883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-29e13c70-d203-4665-b46e-41b0c6ebdd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:51.928363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160b012805e34626b213f728100f7add', 'request_id': 'req-29fa820a-b023-457f-877f-8344de4a9887'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:51.936241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160b012805e34626b213f728100f7add', 'request_id': 'req-29fa820a-b023-457f-877f-8344de4a9887'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:51.945007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160b012805e34626b213f728100f7add', 'request_id': 'req-29fa820a-b023-457f-877f-8344de4a9887'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:51.963404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] The token cache is being invalidate because project 160b012805e34626b213f728100f7add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:51.963601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:51.964356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29fa820a-b023-457f-877f-8344de4a9887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:51.965984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 898/1775] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:51 2022] DELETE /identity/v3/projects/160b012805e34626b213f728100f7add => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:37:51.970824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4dab5-9e37-403f-b9be-672e81393f17 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:51.971186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4dab5-9e37-403f-b9be-672e81393f17 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:51.972124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4dab5-9e37-403f-b9be-672e81393f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.033298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-68c4dab5-9e37-403f-b9be-672e81393f17 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:52.034999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-68c4dab5-9e37-403f-b9be-672e81393f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:52.063944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e13c70-d203-4665-b46e-41b0c6ebdd22 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:52.135691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 899/1776] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:52.145835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f641d9d-df3e-4367-af9b-bf3f7bf2eb2e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:52.146164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f641d9d-df3e-4367-af9b-bf3f7bf2eb2e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:52.146485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f641d9d-df3e-4367-af9b-bf3f7bf2eb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.176375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8f641d9d-df3e-4367-af9b-bf3f7bf2eb2e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:52.178836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:52.178979 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8f641d9d-df3e-4367-af9b-bf3f7bf2eb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:52.179749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:52.180727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:52.181154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.188271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:52.197333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 900/1777] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:52.199848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e3405-2145-465f-8534-b632aeb54ca1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:52.200133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e3405-2145-465f-8534-b632aeb54ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:52.200406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e3405-2145-465f-8534-b632aeb54ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.236224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6d1e3405-2145-465f-8534-b632aeb54ca1 None None] MFA Rules not processed for user `9cecae4323404bd29fe8c05a704de273`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:52.237740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6d1e3405-2145-465f-8534-b632aeb54ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:52.269091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:52.335796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:52.374639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:52.376077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:52.394272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 878/1778] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:52.395126 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:52.407221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 901/1779] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:52.413684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a692d-266c-4694-b1fd-b6da58364a20 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:52.413992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a692d-266c-4694-b1fd-b6da58364a20 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:52.415734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a692d-266c-4694-b1fd-b6da58364a20 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.417956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 902/1780] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 10 13:37:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:37:52.425958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31d7a0-b83a-4dbf-a272-74b6ae2d321e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:52.426153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31d7a0-b83a-4dbf-a272-74b6ae2d321e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:52.426344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31d7a0-b83a-4dbf-a272-74b6ae2d321e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.430084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b204b-09a5-4a9f-b7c5-8addcd531c38 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:52.440545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:52.441856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:52.443080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:52.443446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.445372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:52.445372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:52.450732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:52.452700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9f31d7a0-b83a-4dbf-a272-74b6ae2d321e None None] MFA Rules not processed for user `234eed198ab84e95ae1a7f03785a1481`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:52.454525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9f31d7a0-b83a-4dbf-a272-74b6ae2d321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:52.498470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 879/1781] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:52.500191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:52.559092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e1ed5-0cf7-45f3-a8ed-dfe4a05d0823 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:52.594019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886d91-54f0-42a3-adee-97cf25bed1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:52.594662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886d91-54f0-42a3-adee-97cf25bed1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:52.594940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886d91-54f0-42a3-adee-97cf25bed1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:52.595212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886d91-54f0-42a3-adee-97cf25bed1f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.595849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce886d91-54f0-42a3-adee-97cf25bed1f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:52.599082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce886d91-54f0-42a3-adee-97cf25bed1f6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:52.607468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 903/1782] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:37:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:52.625540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31d7a0-b83a-4dbf-a272-74b6ae2d321e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:52.625540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9f31d7a0-b83a-4dbf-a272-74b6ae2d321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:52.703729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31d7a0-b83a-4dbf-a272-74b6ae2d321e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:52.717224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d551a07-239d-4536-855d-8df9f2e32198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:52.717883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551a07-239d-4536-855d-8df9f2e32198 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:52.718774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551a07-239d-4536-855d-8df9f2e32198 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:52.718774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551a07-239d-4536-855d-8df9f2e32198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:52.740269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d551a07-239d-4536-855d-8df9f2e32198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:52.745051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d551a07-239d-4536-855d-8df9f2e32198 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:52.751937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 880/1783] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:52 2022] POST /identity/v3/projects => generated 361 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:52.756152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886d91-54f0-42a3-adee-97cf25bed1f6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:52.810490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 904/1784] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:37:53.021395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886d91-54f0-42a3-adee-97cf25bed1f6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:53.045048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6053db8-0f9b-43cd-bf2c-19a07f6a7bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:53.046070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6053db8-0f9b-43cd-bf2c-19a07f6a7bec admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:53.047749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6053db8-0f9b-43cd-bf2c-19a07f6a7bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:53.048043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6053db8-0f9b-43cd-bf2c-19a07f6a7bec admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:53.056974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6053db8-0f9b-43cd-bf2c-19a07f6a7bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:53.080833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6053db8-0f9b-43cd-bf2c-19a07f6a7bec admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:53.168083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 881/1785] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:52 2022] POST /identity/v3/users => generated 281 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:53.172402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d551a07-239d-4536-855d-8df9f2e32198 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:53.190226 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d551a07-239d-4536-855d-8df9f2e32198 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:53.232414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46a57f-0204-45ef-b944-bb3fc41b1e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:53.236637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a57f-0204-45ef-b944-bb3fc41b1e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:53.241135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a57f-0204-45ef-b944-bb3fc41b1e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:53.241135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a57f-0204-45ef-b944-bb3fc41b1e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:53.241372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46a57f-0204-45ef-b944-bb3fc41b1e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:53.245455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46a57f-0204-45ef-b944-bb3fc41b1e8b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:53.288715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6053db8-0f9b-43cd-bf2c-19a07f6a7bec admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:53.302148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 905/1786] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:53.304567 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6053db8-0f9b-43cd-bf2c-19a07f6a7bec admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:53.339683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a3f9a-3296-4937-badb-fcb1c697c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:53.340558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3f9a-3296-4937-badb-fcb1c697c70d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:53.342871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718e194-6c01-48bb-9ad8-d3c7158729c4 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:53.343382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718e194-6c01-48bb-9ad8-d3c7158729c4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:53.343801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718e194-6c01-48bb-9ad8-d3c7158729c4 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:53.345445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 906/1787] 10.222.0.52 () {54 vars in 961 bytes} [Wed Aug 10 13:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 13:37:53.345812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3f9a-3296-4937-badb-fcb1c697c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:53.346306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3f9a-3296-4937-badb-fcb1c697c70d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:53.352393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ffe21-338b-43e6-9520-4ce8f32799ef None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:53.352923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ffe21-338b-43e6-9520-4ce8f32799ef None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:53.353442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ffe21-338b-43e6-9520-4ce8f32799ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:53.356904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a3f9a-3296-4937-badb-fcb1c697c70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:53.374731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a3f9a-3296-4937-badb-fcb1c697c70d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:53.404476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 882/1788] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:37:53.410189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a3f9a-3296-4937-badb-fcb1c697c70d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:53.443845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-be3ffe21-338b-43e6-9520-4ce8f32799ef None None] MFA Rules not processed for user `e6a31a4ad1f947308df7e4de047e85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:53.446975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a3f9a-3296-4937-badb-fcb1c697c70d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:53.450264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-be3ffe21-338b-43e6-9520-4ce8f32799ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:53.464328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77ddb4-836b-49f9-b026-f459053c4e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:53.465518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ddb4-836b-49f9-b026-f459053c4e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:53.467823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ddb4-836b-49f9-b026-f459053c4e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:53.468275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ddb4-836b-49f9-b026-f459053c4e96 admin admin] PATH_INFO: `/v3/projects/692c3de6ed894b84b4ea38d0e9cc9cb3/users/1e580e29b5f746d8add0e27220096d14/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:53.520003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77ddb4-836b-49f9-b026-f459053c4e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=692c3de6ed894b84b4ea38d0e9cc9cb3, user_id=1e580e29b5f746d8add0e27220096d14, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:53.524831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77ddb4-836b-49f9-b026-f459053c4e96 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:53.589917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 883/1789] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:53 2022] PUT /identity/v3/projects/692c3de6ed894b84b4ea38d0e9cc9cb3/users/1e580e29b5f746d8add0e27220096d14/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:53.591329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aff028-a70f-4046-905c-6092a7e34f71 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:53.591614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aff028-a70f-4046-905c-6092a7e34f71 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:53.591889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aff028-a70f-4046-905c-6092a7e34f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:53.693157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-37aff028-a70f-4046-905c-6092a7e34f71 None None] MFA Rules not processed for user `1e580e29b5f746d8add0e27220096d14`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:53.698418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-37aff028-a70f-4046-905c-6092a7e34f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:53.721862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 907/1790] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Aug 10 13:37:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:53.728188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ffe21-338b-43e6-9520-4ce8f32799ef None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:53.770097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ffe21-338b-43e6-9520-4ce8f32799ef None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:53.882169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:54.002801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 884/1791] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:54.004639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d424-5878-4375-9bfd-176abf5219eb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:54.005018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d424-5878-4375-9bfd-176abf5219eb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:54.005449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d424-5878-4375-9bfd-176abf5219eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:54.025554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:54.040825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3fe83-2058-4e80-8a9d-8016d5c591b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:54.041791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fe83-2058-4e80-8a9d-8016d5c591b7 service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:54.045867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fe83-2058-4e80-8a9d-8016d5c591b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:54.046344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fe83-2058-4e80-8a9d-8016d5c591b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:54.050262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-05f3fe83-2058-4e80-8a9d-8016d5c591b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:54.077702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e888d424-5878-4375-9bfd-176abf5219eb None None] MFA Rules not processed for user `5573338f0bd64c729c7d89345069ca6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:54.080831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e888d424-5878-4375-9bfd-176abf5219eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:54.086801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3fe83-2058-4e80-8a9d-8016d5c591b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:54.088803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3fe83-2058-4e80-8a9d-8016d5c591b7 service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:54.118639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 908/1792] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:54.119980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb0f89-0388-43ae-8ab1-bba7555a7460 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:54.120267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb0f89-0388-43ae-8ab1-bba7555a7460 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:54.120541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb0f89-0388-43ae-8ab1-bba7555a7460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:54.184089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ecdb0f89-0388-43ae-8ab1-bba7555a7460 None None] MFA Rules not processed for user `289c7d8b89cd445a924f5ac07e85fe48`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:54.185346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ecdb0f89-0388-43ae-8ab1-bba7555a7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:54.347746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 885/1793] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:54.366014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc11e83-27e9-4953-af16-0e928988cc74 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:54.367870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc11e83-27e9-4953-af16-0e928988cc74 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:54.369613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc11e83-27e9-4953-af16-0e928988cc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:54.426515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bbc11e83-27e9-4953-af16-0e928988cc74 None None] MFA Rules not processed for user `5573338f0bd64c729c7d89345069ca6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:54.429557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bbc11e83-27e9-4953-af16-0e928988cc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:54.455348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 886/1794] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:54.456579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc11e83-27e9-4953-af16-0e928988cc74 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:54.458811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bbc11e83-27e9-4953-af16-0e928988cc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:54.518912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 909/1795] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:54.532223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb0f89-0388-43ae-8ab1-bba7555a7460 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:54.540291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ecdb0f89-0388-43ae-8ab1-bba7555a7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:54.576980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc11e83-27e9-4953-af16-0e928988cc74 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:54.582991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d823cc5-db95-4a3f-9c2c-99383c7cfdc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:54.583827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d823cc5-db95-4a3f-9c2c-99383c7cfdc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:54.584204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d823cc5-db95-4a3f-9c2c-99383c7cfdc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:54.584593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d823cc5-db95-4a3f-9c2c-99383c7cfdc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:54.609837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d823cc5-db95-4a3f-9c2c-99383c7cfdc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:54.616596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d823cc5-db95-4a3f-9c2c-99383c7cfdc1 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:54.676824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 887/1796] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:54.680938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d823cc5-db95-4a3f-9c2c-99383c7cfdc1 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:54.717674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb0f89-0388-43ae-8ab1-bba7555a7460 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:54.729358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:54.730175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:54.730600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:54.731013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:54.735120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:54.850118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d823cc5-db95-4a3f-9c2c-99383c7cfdc1 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:54.860501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:54.863810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:54.864928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:54.865339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] PATH_INFO: `/v3/users/1e580e29b5f746d8add0e27220096d14` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:54.871942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:54.872575 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e580e29b5f746d8add0e27220096d14)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:54.874251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:54.876142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:54.885691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:54.896637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 910/1797] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:54.899402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:54.903896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:55.017814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e580e29b5f746d8add0e27220096d14'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:55.031439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:55.032926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e580e29b5f746d8add0e27220096d14'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:55.034361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e580e29b5f746d8add0e27220096d14'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:55.056203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e580e29b5f746d8add0e27220096d14'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:55.088307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c1c1c278-e8ea-4fe2-bbc8-f17d6d5cb081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e580e29b5f746d8add0e27220096d14'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:55.104959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 888/1798] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:54 2022] DELETE /identity/v3/users/1e580e29b5f746d8add0e27220096d14 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:55.107133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259f49-ea90-4867-9233-767124892eaf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:55.107418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259f49-ea90-4867-9233-767124892eaf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:55.107691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259f49-ea90-4867-9233-767124892eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:55.153038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d2259f49-ea90-4867-9233-767124892eaf None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:55.156450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d2259f49-ea90-4867-9233-767124892eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:55.271641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d4804-c4b3-4e23-88cf-41616e07f142 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:55.331566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 889/1799] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:55.345105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259f49-ea90-4867-9233-767124892eaf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:55.346812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d2259f49-ea90-4867-9233-767124892eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:55.416317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:55.416685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:55.417156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:55.417488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:55.419413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:55.471901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259f49-ea90-4867-9233-767124892eaf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:55.479917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:55.480800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:55.481366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:55.482334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:55.505068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:55.506822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:55.529874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:55.531446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:55.535067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 890/1800] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:55.551519 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 911/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:56.383357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:56.384897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:56.425024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b9555-6aec-4a10-901a-c9ed62a60a7d service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:56.449331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e84cb-f510-44dc-85dd-a31c453fe8fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:56.450343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e84cb-f510-44dc-85dd-a31c453fe8fd service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:56.450807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e84cb-f510-44dc-85dd-a31c453fe8fd service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:56.451227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e84cb-f510-44dc-85dd-a31c453fe8fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:56.459863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7a9e84cb-f510-44dc-85dd-a31c453fe8fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:56.528910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:56.645360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e84cb-f510-44dc-85dd-a31c453fe8fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:56.652143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e84cb-f510-44dc-85dd-a31c453fe8fd service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:56.683407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 891/1802] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:56.732126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:56.733701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:56.791818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529bf96-6a5f-47fc-a05c-9449cc346010 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:56.807090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:56.807946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:56.809258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:56.810322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] PATH_INFO: `/v3/users/109e3debf81b49c5be41474ee7b10aa6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:56.814356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=109e3debf81b49c5be41474ee7b10aa6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:56.819435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:56.943769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109e3debf81b49c5be41474ee7b10aa6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:56.963611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109e3debf81b49c5be41474ee7b10aa6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:56.964305 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109e3debf81b49c5be41474ee7b10aa6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:56.989464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:57.011019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109e3debf81b49c5be41474ee7b10aa6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:57.012283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109e3debf81b49c5be41474ee7b10aa6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:57.033488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 912/1803] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:37:56 2022] DELETE /identity/v3/users/109e3debf81b49c5be41474ee7b10aa6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:57.037817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d2624-95fc-40c3-a403-3215614989a4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:57.038100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d2624-95fc-40c3-a403-3215614989a4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:57.038748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d2624-95fc-40c3-a403-3215614989a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:57.088078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-498d2624-95fc-40c3-a403-3215614989a4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:57.093534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-498d2624-95fc-40c3-a403-3215614989a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:57.286472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 892/1804] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:57.339510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:57.341778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:57.634706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870f79f-1ac5-4e9e-85ff-8dd05720fe1e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:57.656823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4cb5e-2f90-4836-a836-eb187a99b5ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:57.657442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4cb5e-2f90-4836-a836-eb187a99b5ab service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:57.657732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4cb5e-2f90-4836-a836-eb187a99b5ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:57.657997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4cb5e-2f90-4836-a836-eb187a99b5ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:57.695152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4cb5e-2f90-4836-a836-eb187a99b5ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:57.699918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4cb5e-2f90-4836-a836-eb187a99b5ab service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:57.735920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 913/1805] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:37:57.739571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d2624-95fc-40c3-a403-3215614989a4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:57.741164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-498d2624-95fc-40c3-a403-3215614989a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:57.777003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d2624-95fc-40c3-a403-3215614989a4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:57.798597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:57.802611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:57.804112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:57.804658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] PATH_INFO: `/v3/projects/692c3de6ed894b84b4ea38d0e9cc9cb3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:57.808533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=692c3de6ed894b84b4ea38d0e9cc9cb3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:57.820921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:58.031581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '692c3de6ed894b84b4ea38d0e9cc9cb3', 'request_id': 'req-17cba93f-8c8f-4c56-938f-be3f53e26071'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:58.045877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '692c3de6ed894b84b4ea38d0e9cc9cb3', 'request_id': 'req-17cba93f-8c8f-4c56-938f-be3f53e26071'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:58.056904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '692c3de6ed894b84b4ea38d0e9cc9cb3', 'request_id': 'req-17cba93f-8c8f-4c56-938f-be3f53e26071'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:58.096612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] The token cache is being invalidate because project 692c3de6ed894b84b4ea38d0e9cc9cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:37:58.101095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:58.103619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-17cba93f-8c8f-4c56-938f-be3f53e26071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:37:58.130055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 893/1806] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:37:57 2022] DELETE /identity/v3/projects/692c3de6ed894b84b4ea38d0e9cc9cb3 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:58.135284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b1f3-6803-4e32-8211-a3b85c5e28fc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:58.135533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b1f3-6803-4e32-8211-a3b85c5e28fc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:58.135808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b1f3-6803-4e32-8211-a3b85c5e28fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:58.204607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-43b5b1f3-6803-4e32-8211-a3b85c5e28fc None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:58.218990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-43b5b1f3-6803-4e32-8211-a3b85c5e28fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:58.296639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a58b7-3ac3-4d6b-a2b6-3c3abf88439b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:58.296938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a58b7-3ac3-4d6b-a2b6-3c3abf88439b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:58.297193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a58b7-3ac3-4d6b-a2b6-3c3abf88439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:58.444936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 914/1807] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:58.454828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b7e4-81af-4bea-8faf-b3d89783410c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:58.455110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b7e4-81af-4bea-8faf-b3d89783410c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:58.455379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b7e4-81af-4bea-8faf-b3d89783410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:58.455994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2f2a58b7-3ac3-4d6b-a2b6-3c3abf88439b None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:58.461804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2f2a58b7-3ac3-4d6b-a2b6-3c3abf88439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:58.495578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6211b7e4-81af-4bea-8faf-b3d89783410c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:58.496845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6211b7e4-81af-4bea-8faf-b3d89783410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:58.533519 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 915/1808] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:37:58.538099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211b7e4-81af-4bea-8faf-b3d89783410c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:58.552020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211b7e4-81af-4bea-8faf-b3d89783410c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:58.590603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:58.591245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:58.592369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:58.592662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:58.593367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:37:58.593677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:58.594377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:58.595315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:58.652993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 916/1809] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:37:58.669544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:58.686022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa48eb8-f169-4d54-90a0-0c74d23a5296 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:58.709375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df731c9-4270-4753-afb0-86e4b2bdc919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:58.710502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df731c9-4270-4753-afb0-86e4b2bdc919 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:58.711021 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df731c9-4270-4753-afb0-86e4b2bdc919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:58.711300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df731c9-4270-4753-afb0-86e4b2bdc919 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:58.713417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df731c9-4270-4753-afb0-86e4b2bdc919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:58.715961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df731c9-4270-4753-afb0-86e4b2bdc919 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:58.819841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 917/1810] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:37:58 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:58.826806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd604-e607-4979-955e-e29c9c5f0fed None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:58.827384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd604-e607-4979-955e-e29c9c5f0fed None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:58.828331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd604-e607-4979-955e-e29c9c5f0fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:58.846711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 894/1811] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:37:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:58.848937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a58b7-3ac3-4d6b-a2b6-3c3abf88439b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:58.915395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a58b7-3ac3-4d6b-a2b6-3c3abf88439b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:59.006729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-12cbd604-e607-4979-955e-e29c9c5f0fed None None] MFA Rules not processed for user `5573338f0bd64c729c7d89345069ca6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:59.018096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-12cbd604-e607-4979-955e-e29c9c5f0fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:59.140625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:59.151972 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:59.152263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:59.152691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:59.153534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:59.160165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:59.289906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 895/1812] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:37:58 2022] POST /identity/v3/users => generated 287 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:37:59.296659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:59.298407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:59.306483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:37:59.391905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 918/1813] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 570 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:37:59.398131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbd604-e607-4979-955e-e29c9c5f0fed None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:59.448050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbd604-e607-4979-955e-e29c9c5f0fed None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:59.473973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001390b-11bc-46e4-8416-950ec7b78d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:59.480349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001390b-11bc-46e4-8416-950ec7b78d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:59.480662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001390b-11bc-46e4-8416-950ec7b78d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:59.483363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001390b-11bc-46e4-8416-950ec7b78d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:59.486060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001390b-11bc-46e4-8416-950ec7b78d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:59.497497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001390b-11bc-46e4-8416-950ec7b78d62 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:59.536144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 919/1814] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:59.546778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001390b-11bc-46e4-8416-950ec7b78d62 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:59.582383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945c8ed-0a39-4e8e-93b4-3e833f4fc2f0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:59.592051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001390b-11bc-46e4-8416-950ec7b78d62 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:59.598208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bcbda-645f-41b1-a123-3d96147d63c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:59.598854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcbda-645f-41b1-a123-3d96147d63c1 service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:59.599257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcbda-645f-41b1-a123-3d96147d63c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:59.599538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcbda-645f-41b1-a123-3d96147d63c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:59.606465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000569f-0bab-4b5f-ba29-f38644702f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:59.607224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000569f-0bab-4b5f-ba29-f38644702f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:59.607608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000569f-0bab-4b5f-ba29-f38644702f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:59.607982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000569f-0bab-4b5f-ba29-f38644702f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:59.608734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000569f-0bab-4b5f-ba29-f38644702f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:59.610757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000569f-0bab-4b5f-ba29-f38644702f81 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:59.652366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 920/1815] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:37:59.656891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:37:59.658137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000569f-0bab-4b5f-ba29-f38644702f81 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:37:59.691153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bcbda-645f-41b1-a123-3d96147d63c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:59.692510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000569f-0bab-4b5f-ba29-f38644702f81 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:37:59.692959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bcbda-645f-41b1-a123-3d96147d63c1 service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:59.714170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03009ae7-1968-49d1-95e8-ca3628591d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:37:59.715101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03009ae7-1968-49d1-95e8-ca3628591d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:59.716056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03009ae7-1968-49d1-95e8-ca3628591d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:59.718905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 896/1816] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:37:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:37:59.720815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03009ae7-1968-49d1-95e8-ca3628591d8f admin admin] PATH_INFO: `/v3/projects/bb02136e4dc64626adcef53a0c44fab9/users/1886bd370a034ce49b337ecb981cf7d4/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:59.780639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03009ae7-1968-49d1-95e8-ca3628591d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb02136e4dc64626adcef53a0c44fab9, user_id=1886bd370a034ce49b337ecb981cf7d4, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:37:59.783990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03009ae7-1968-49d1-95e8-ca3628591d8f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:37:59.825912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 921/1817] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:37:59 2022] PUT /identity/v3/projects/bb02136e4dc64626adcef53a0c44fab9/users/1886bd370a034ce49b337ecb981cf7d4/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:37:59.837046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822d2cd-84a8-4dd2-b476-d0d734e3a281 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:37:59.837408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822d2cd-84a8-4dd2-b476-d0d734e3a281 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:37:59.837686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822d2cd-84a8-4dd2-b476-d0d734e3a281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:37:59.952324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4822d2cd-84a8-4dd2-b476-d0d734e3a281 None None] MFA Rules not processed for user `1886bd370a034ce49b337ecb981cf7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:37:59.954352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4822d2cd-84a8-4dd2-b476-d0d734e3a281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:00.098616 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 897/1818] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:37:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:00.104166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03009ae7-1968-49d1-95e8-ca3628591d8f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:00.348836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03009ae7-1968-49d1-95e8-ca3628591d8f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:00.370672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc70c52-fb8c-4bda-b111-d348929b2cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:00.371555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc70c52-fb8c-4bda-b111-d348929b2cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:00.371997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc70c52-fb8c-4bda-b111-d348929b2cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:00.372471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc70c52-fb8c-4bda-b111-d348929b2cd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:00.373457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc70c52-fb8c-4bda-b111-d348929b2cd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:00.376661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc70c52-fb8c-4bda-b111-d348929b2cd6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:00.471427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 922/1819] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:00 2022] POST /identity/v3/projects => generated 375 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:00.472400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822d2cd-84a8-4dd2-b476-d0d734e3a281 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:00.492676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822d2cd-84a8-4dd2-b476-d0d734e3a281 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:00.691188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb0bb4-1eb1-4edd-bde0-cb9691f573df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:00.691834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb0bb4-1eb1-4edd-bde0-cb9691f573df admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:00.692221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb0bb4-1eb1-4edd-bde0-cb9691f573df admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:00.692493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb0bb4-1eb1-4edd-bde0-cb9691f573df admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:00.693123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb0bb4-1eb1-4edd-bde0-cb9691f573df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:00.695647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb0bb4-1eb1-4edd-bde0-cb9691f573df admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:00.753795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 898/1820] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:00 2022] POST /identity/v3/users => generated 294 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:38:00.754777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc70c52-fb8c-4bda-b111-d348929b2cd6 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:00.771864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc70c52-fb8c-4bda-b111-d348929b2cd6 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:00.792768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c2e1f-9379-4e56-a625-af71f1f58482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:00.793492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c2e1f-9379-4e56-a625-af71f1f58482 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:00.793771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c2e1f-9379-4e56-a625-af71f1f58482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:00.794040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c2e1f-9379-4e56-a625-af71f1f58482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:00.794696 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c2e1f-9379-4e56-a625-af71f1f58482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:00.797802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c2e1f-9379-4e56-a625-af71f1f58482 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:00.834641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 923/1821] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:38:00.837948 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb0bb4-1eb1-4edd-bde0-cb9691f573df admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:00.859354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb0bb4-1eb1-4edd-bde0-cb9691f573df admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:00.869540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696a86d-47e1-466c-8cb5-336c896d4831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:00.873009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a86d-47e1-466c-8cb5-336c896d4831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:00.875379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a86d-47e1-466c-8cb5-336c896d4831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:00.886730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a86d-47e1-466c-8cb5-336c896d4831 admin admin] PATH_INFO: `/v3/projects/ba760e2eb0f54dcc9c7e05a0c63e42ec/users/6e2096e9095449748d31a3765a97a7cc/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:00.953874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696a86d-47e1-466c-8cb5-336c896d4831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba760e2eb0f54dcc9c7e05a0c63e42ec, user_id=6e2096e9095449748d31a3765a97a7cc, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:00.962642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696a86d-47e1-466c-8cb5-336c896d4831 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:01.009243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 899/1822] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:00 2022] PUT /identity/v3/projects/ba760e2eb0f54dcc9c7e05a0c63e42ec/users/6e2096e9095449748d31a3765a97a7cc/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:01.010476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c2e1f-9379-4e56-a625-af71f1f58482 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:01.038898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c2e1f-9379-4e56-a625-af71f1f58482 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:01.240240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3883b4-ac25-4c52-8453-7a79448a7e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:01.240862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3883b4-ac25-4c52-8453-7a79448a7e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:01.241477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3883b4-ac25-4c52-8453-7a79448a7e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:01.241973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3883b4-ac25-4c52-8453-7a79448a7e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:01.244278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3883b4-ac25-4c52-8453-7a79448a7e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:01.246470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3883b4-ac25-4c52-8453-7a79448a7e92 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:01.262236 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 924/1823] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:01.263449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696a86d-47e1-466c-8cb5-336c896d4831 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:01.280955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696a86d-47e1-466c-8cb5-336c896d4831 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:01.287538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:01.289403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:01.290903 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:01.292097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] PATH_INFO: `/v3/projects/ba760e2eb0f54dcc9c7e05a0c63e42ec/users/6e2096e9095449748d31a3765a97a7cc/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:01.308419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba760e2eb0f54dcc9c7e05a0c63e42ec, user_id=6e2096e9095449748d31a3765a97a7cc, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:01.315188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:01.358810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 900/1824] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:01 2022] PUT /identity/v3/projects/ba760e2eb0f54dcc9c7e05a0c63e42ec/users/6e2096e9095449748d31a3765a97a7cc/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:01.360018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8c7b-5f1c-400b-b5d6-85a2726f11c2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:01.360492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8c7b-5f1c-400b-b5d6-85a2726f11c2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:01.360939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8c7b-5f1c-400b-b5d6-85a2726f11c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:01.391780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:01.395084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:01.426781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:01.431830 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:01.452095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9c9d8c7b-5f1c-400b-b5d6-85a2726f11c2 None None] MFA Rules not processed for user `6e2096e9095449748d31a3765a97a7cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:01.453356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9c9d8c7b-5f1c-400b-b5d6-85a2726f11c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:01.602513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b6272-e557-4ce1-a62a-18dc01ee2466 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:01.608236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:01.609309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:01.609968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:01.610445 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:01.612990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:01.678717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 925/1825] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:01.680683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bf571-684a-4cbe-a135-655304abd66e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:01.681023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bf571-684a-4cbe-a135-655304abd66e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:01.681302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bf571-684a-4cbe-a135-655304abd66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:01.722985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e82bf571-684a-4cbe-a135-655304abd66e None None] MFA Rules not processed for user `234eed198ab84e95ae1a7f03785a1481`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:01.724227 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e82bf571-684a-4cbe-a135-655304abd66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:01.772232 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:01.774573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:01.821101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 901/1826] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:01.822142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:01.830944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:01.941264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 926/1827] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Aug 10 13:38:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:01.941264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bf571-684a-4cbe-a135-655304abd66e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:01.946398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e82bf571-684a-4cbe-a135-655304abd66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:01.988858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bf571-684a-4cbe-a135-655304abd66e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:02.030061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:02.031983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:02.032325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:02.032596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:02.104124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e17bb4-8238-4a31-864e-6c267f7d38e7 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:02.118248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:02.118898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:02.119172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:02.119431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] PATH_INFO: `/v3/projects/d44a6bc2060c41eaacf8fa27aaf0ef5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:02.122333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d44a6bc2060c41eaacf8fa27aaf0ef5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:02.124403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:02.237818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:02.250281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:02.263999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd44a6bc2060c41eaacf8fa27aaf0ef5d', 'request_id': 'req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:02.281954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd44a6bc2060c41eaacf8fa27aaf0ef5d', 'request_id': 'req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:02.300329 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd44a6bc2060c41eaacf8fa27aaf0ef5d', 'request_id': 'req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:02.307403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 927/1828] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:02.310759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:02.316543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:02.321073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:02.344839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] The token cache is being invalidate because project d44a6bc2060c41eaacf8fa27aaf0ef5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:02.349853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:02.354298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:02.358606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 902/1829] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:01 2022] DELETE /identity/v3/projects/d44a6bc2060c41eaacf8fa27aaf0ef5d => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:02.363076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:02.372691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ad579-6baa-487d-9b64-9b841852e85f service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:02.377805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:02.551799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:02.582400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:02.583292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:02.583618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:02.583849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:02.592001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:02.640495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:02.642041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:02.690755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 928/1830] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:02.727440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd6100-e12d-4fe8-9ff6-7d10fd62b6ae admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:02.749120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:02.749863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:02.750141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:02.750405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] PATH_INFO: `/v3/users/233196973602453bad2efb14ec905ded` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:02.764476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=233196973602453bad2efb14ec905ded)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:02.766542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:02.953561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '233196973602453bad2efb14ec905ded'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:02.980382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '233196973602453bad2efb14ec905ded'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:02.984772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '233196973602453bad2efb14ec905ded'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:02.999548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '233196973602453bad2efb14ec905ded'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:03.055868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-df4f9ac1-b2c5-4c54-a4f2-4a7ca5b3b3b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '233196973602453bad2efb14ec905ded'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:03.071665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 903/1831] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:02 2022] DELETE /identity/v3/users/233196973602453bad2efb14ec905ded => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:05.193767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:05.218676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112f80-2853-443d-b51d-5ec3beeda272 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:05.449026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:05.449884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:05.450430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:05.450772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] PATH_INFO: `/v3/projects/fc0b9f9d151840448b9e440429ef776b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:05.452587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc0b9f9d151840448b9e440429ef776b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:05.456026 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:05.575861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc0b9f9d151840448b9e440429ef776b', 'request_id': 'req-4e78327c-20a9-4fc8-aa6e-23b29023ed40'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:05.597178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc0b9f9d151840448b9e440429ef776b', 'request_id': 'req-4e78327c-20a9-4fc8-aa6e-23b29023ed40'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:05.607936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc0b9f9d151840448b9e440429ef776b', 'request_id': 'req-4e78327c-20a9-4fc8-aa6e-23b29023ed40'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:05.640463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] The token cache is being invalidate because project fc0b9f9d151840448b9e440429ef776b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:05.640794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:05.641732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e78327c-20a9-4fc8-aa6e-23b29023ed40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:05.643662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 929/1832] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:05 2022] DELETE /identity/v3/projects/fc0b9f9d151840448b9e440429ef776b => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:05.649123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f758993-fb4a-4701-8dfb-d60f0e786e8f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:05.650185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f758993-fb4a-4701-8dfb-d60f0e786e8f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:05.650794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f758993-fb4a-4701-8dfb-d60f0e786e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:05.677709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2f758993-fb4a-4701-8dfb-d60f0e786e8f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:05.679382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2f758993-fb4a-4701-8dfb-d60f0e786e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:05.772926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 904/1833] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:05.783246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eb85d-70bb-4474-b713-d706ae11f4ec None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:05.783673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eb85d-70bb-4474-b713-d706ae11f4ec None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:05.783844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eb85d-70bb-4474-b713-d706ae11f4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:05.821015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5d5eb85d-70bb-4474-b713-d706ae11f4ec None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:05.824704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5d5eb85d-70bb-4474-b713-d706ae11f4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:05.856252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 930/1834] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:05.858286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f758993-fb4a-4701-8dfb-d60f0e786e8f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:05.885916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f758993-fb4a-4701-8dfb-d60f0e786e8f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:05.923000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:05.926301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:05.928091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:05.929892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:05.934142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:05.935314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:05.945418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:06.028278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 905/1835] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:38:06.030116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eb85d-70bb-4474-b713-d706ae11f4ec None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:06.048577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eb85d-70bb-4474-b713-d706ae11f4ec None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:06.064939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:06.066958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:06.082482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fe4f4-cf82-41ac-8f89-e3debe60aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:06.086906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fe4f4-cf82-41ac-8f89-e3debe60aa53 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.089948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fe4f4-cf82-41ac-8f89-e3debe60aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.092861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:06.093675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fe4f4-cf82-41ac-8f89-e3debe60aa53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.094354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fe4f4-cf82-41ac-8f89-e3debe60aa53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:06.110067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fe4f4-cf82-41ac-8f89-e3debe60aa53 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:06.119387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dfb51-25ee-4f3b-a4ff-3c85abc84af6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:06.160099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:06.160455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.162375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.163340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] PATH_INFO: `/v3/users/1886bd370a034ce49b337ecb981cf7d4` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.185318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 931/1836] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:06 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:06.185809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1886bd370a034ce49b337ecb981cf7d4)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:06.188857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:06.191048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e134-2938-4b11-b9f5-317b8801302e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.191547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e134-2938-4b11-b9f5-317b8801302e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.191995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e134-2938-4b11-b9f5-317b8801302e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.197672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 932/1837] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:38:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:38:06.198541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e134-2938-4b11-b9f5-317b8801302e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:06.313481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:06.343708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1886bd370a034ce49b337ecb981cf7d4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:06.350017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e134-2938-4b11-b9f5-317b8801302e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:06.357136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1886bd370a034ce49b337ecb981cf7d4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:06.359593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9237b3-bc74-41d0-8040-27fa9956c70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:06.360229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9237b3-bc74-41d0-8040-27fa9956c70c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.360507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9237b3-bc74-41d0-8040-27fa9956c70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.360780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9237b3-bc74-41d0-8040-27fa9956c70c admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.361260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1886bd370a034ce49b337ecb981cf7d4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:06.361450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9237b3-bc74-41d0-8040-27fa9956c70c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:06.363653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9237b3-bc74-41d0-8040-27fa9956c70c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:06.380272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1886bd370a034ce49b337ecb981cf7d4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:06.422510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1886bd370a034ce49b337ecb981cf7d4'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:06.441457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 933/1838] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:06 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:06.441457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b02db-4b9f-48df-8a47-bea099aac9a5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.441457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b02db-4b9f-48df-8a47-bea099aac9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.441457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b02db-4b9f-48df-8a47-bea099aac9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.441854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 906/1839] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:06 2022] DELETE /identity/v3/users/1886bd370a034ce49b337ecb981cf7d4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:06.442724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:06.447958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:06.536678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-913b02db-4b9f-48df-8a47-bea099aac9a5 None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:06.538158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-913b02db-4b9f-48df-8a47-bea099aac9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:06.561457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c2b00-8728-43fe-ba79-a843bfda0515 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:06.570815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208293ee-546a-4d28-9c8c-90220e69f190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:06.571530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208293ee-546a-4d28-9c8c-90220e69f190 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.571871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208293ee-546a-4d28-9c8c-90220e69f190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.572147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208293ee-546a-4d28-9c8c-90220e69f190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.572805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208293ee-546a-4d28-9c8c-90220e69f190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:06.575492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208293ee-546a-4d28-9c8c-90220e69f190 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:06.611534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 907/1840] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:06.616254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8abc8-3d34-408c-8870-b35ff5fa9bf0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.616503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8abc8-3d34-408c-8870-b35ff5fa9bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.617026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8abc8-3d34-408c-8870-b35ff5fa9bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.697638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9cc8abc8-3d34-408c-8870-b35ff5fa9bf0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:06.706718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9cc8abc8-3d34-408c-8870-b35ff5fa9bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:06.748653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 908/1841] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:06.750244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8abc8-3d34-408c-8870-b35ff5fa9bf0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:06.755644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 934/1842] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:06.760190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9cc8abc8-3d34-408c-8870-b35ff5fa9bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:06.763207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b02db-4b9f-48df-8a47-bea099aac9a5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:06.819710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b02db-4b9f-48df-8a47-bea099aac9a5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:06.823231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8abc8-3d34-408c-8870-b35ff5fa9bf0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:06.843491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:06.846177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.848487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:06.848947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.849435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.851101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:06.852967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:06.853382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:06.853799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] PATH_INFO: `/v3/users/bec2211b18a0499db1caeba46670a9d5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:06.861065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:06.861975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec2211b18a0499db1caeba46670a9d5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:06.872600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:06.910058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 935/1843] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:06.911922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:06.926307 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:07.062475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec2211b18a0499db1caeba46670a9d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:07.100405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec2211b18a0499db1caeba46670a9d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:07.101867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec2211b18a0499db1caeba46670a9d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:07.129249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:07.137195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec2211b18a0499db1caeba46670a9d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:07.160003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1c666-75e7-46f2-a7d7-e127e3b7988a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:07.169486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ae1b6-78b4-4f68-ac85-cbf41539761e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:07.170355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ae1b6-78b4-4f68-ac85-cbf41539761e service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:07.170639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ae1b6-78b4-4f68-ac85-cbf41539761e service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:07.170917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ae1b6-78b4-4f68-ac85-cbf41539761e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:07.173012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ab3ae1b6-78b4-4f68-ac85-cbf41539761e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:07.196741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec2211b18a0499db1caeba46670a9d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:07.207700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ae1b6-78b4-4f68-ac85-cbf41539761e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:07.209259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ae1b6-78b4-4f68-ac85-cbf41539761e service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:07.218316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 909/1844] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:06 2022] DELETE /identity/v3/users/bec2211b18a0499db1caeba46670a9d5 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:07.219218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:07.223865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:07.226092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 936/1845] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:07.229069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bfd73-115f-45d7-bb00-c399e7a90101 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:07.229381 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bfd73-115f-45d7-bb00-c399e7a90101 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:07.229688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bfd73-115f-45d7-bb00-c399e7a90101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:07.266649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-cf9bfd73-115f-45d7-bb00-c399e7a90101 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:07.268710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cf9bfd73-115f-45d7-bb00-c399e7a90101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:07.370169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96daff-f62e-4078-8517-614a22f10c79 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:07.382133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144bf60-8a72-49aa-a8f1-0db6156eacc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:07.383179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144bf60-8a72-49aa-a8f1-0db6156eacc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:07.383476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144bf60-8a72-49aa-a8f1-0db6156eacc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:07.383756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144bf60-8a72-49aa-a8f1-0db6156eacc1 admin admin] PATH_INFO: `/v3/projects/b0361ee3aa6d46febd2aa21f94188e5a/users/57e2801ab59f4f07acec902d3cfa5f13/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:07.422387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144bf60-8a72-49aa-a8f1-0db6156eacc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0361ee3aa6d46febd2aa21f94188e5a, user_id=57e2801ab59f4f07acec902d3cfa5f13, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:07.425346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144bf60-8a72-49aa-a8f1-0db6156eacc1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:07.438755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 937/1846] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:07.464991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 910/1847] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:07 2022] PUT /identity/v3/projects/b0361ee3aa6d46febd2aa21f94188e5a/users/57e2801ab59f4f07acec902d3cfa5f13/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:07.476022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ee32e-194d-48e5-aee7-b2c3d57d2917 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:07.476353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ee32e-194d-48e5-aee7-b2c3d57d2917 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:07.476608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ee32e-194d-48e5-aee7-b2c3d57d2917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:07.500476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:07.538679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c91ee32e-194d-48e5-aee7-b2c3d57d2917 None None] MFA Rules not processed for user `57e2801ab59f4f07acec902d3cfa5f13`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:07.540440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c91ee32e-194d-48e5-aee7-b2c3d57d2917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:07.682207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 938/1848] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:07.687666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144bf60-8a72-49aa-a8f1-0db6156eacc1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:07.738341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ee32e-194d-48e5-aee7-b2c3d57d2917 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:07.739994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c91ee32e-194d-48e5-aee7-b2c3d57d2917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:07.871178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ee32e-194d-48e5-aee7-b2c3d57d2917 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:07.875903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:07.877352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:07.877748 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:07.878592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:07.927051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144bf60-8a72-49aa-a8f1-0db6156eacc1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:07.942643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1984a52-966f-4baa-8407-8ae5dbbc82bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:07.945751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1984a52-966f-4baa-8407-8ae5dbbc82bc admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:07.946073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1984a52-966f-4baa-8407-8ae5dbbc82bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:07.946351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1984a52-966f-4baa-8407-8ae5dbbc82bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:07.947101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1984a52-966f-4baa-8407-8ae5dbbc82bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:07.953422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1984a52-966f-4baa-8407-8ae5dbbc82bc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:07.994358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:07.998803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:08.022399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 939/1849] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:08.024079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:08.026427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:08.027674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 911/1850] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:07 2022] POST /identity/v3/projects => generated 369 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:08.032100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1984a52-966f-4baa-8407-8ae5dbbc82bc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:08.097098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:08.152633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:08.162119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408af71-501e-402d-b932-050426a52cf9 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:08.194140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1984a52-966f-4baa-8407-8ae5dbbc82bc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:08.213604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab2aa1-e091-48fe-ae62-6d3c2ff3afdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:08.214281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2aa1-e091-48fe-ae62-6d3c2ff3afdb admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:08.214603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2aa1-e091-48fe-ae62-6d3c2ff3afdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:08.215270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2aa1-e091-48fe-ae62-6d3c2ff3afdb admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:08.216013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab2aa1-e091-48fe-ae62-6d3c2ff3afdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:08.223511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab2aa1-e091-48fe-ae62-6d3c2ff3afdb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:08.283923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 912/1851] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:08 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:08.288601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab2aa1-e091-48fe-ae62-6d3c2ff3afdb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:08.314506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab2aa1-e091-48fe-ae62-6d3c2ff3afdb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:08.329528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3765aee-c66b-4f63-8bcd-eadd22751830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:08.331827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3765aee-c66b-4f63-8bcd-eadd22751830 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:08.332274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3765aee-c66b-4f63-8bcd-eadd22751830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:08.333812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3765aee-c66b-4f63-8bcd-eadd22751830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:08.334676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3765aee-c66b-4f63-8bcd-eadd22751830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:08.341280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3765aee-c66b-4f63-8bcd-eadd22751830 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:08.349687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:08.373309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 913/1852] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:08.377974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:08.378348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3765aee-c66b-4f63-8bcd-eadd22751830 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:08.378486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:08.378837 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:08.381866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:08.402194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3765aee-c66b-4f63-8bcd-eadd22751830 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:08.420176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c3d6-75df-4f0e-8288-fab577d6bb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:08.420681 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:08.421246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c3d6-75df-4f0e-8288-fab577d6bb98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:08.422534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c3d6-75df-4f0e-8288-fab577d6bb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:08.423082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c3d6-75df-4f0e-8288-fab577d6bb98 admin admin] PATH_INFO: `/v3/projects/abecf29ba1b64db68cb93a6d817635e5/users/781229479c0e41e39f5bacf2f6db07d9/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:08.425764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:08.472562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c3d6-75df-4f0e-8288-fab577d6bb98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abecf29ba1b64db68cb93a6d817635e5, user_id=781229479c0e41e39f5bacf2f6db07d9, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:08.484298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c3d6-75df-4f0e-8288-fab577d6bb98 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:08.490853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 940/1853] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:08.519732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 914/1854] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:08 2022] PUT /identity/v3/projects/abecf29ba1b64db68cb93a6d817635e5/users/781229479c0e41e39f5bacf2f6db07d9/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:08.524390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:08.547789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b23702-18be-4182-afca-21a8f1fdb3f4 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:08.803894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f391756-477f-41b5-a1bc-1f2de8d834a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:08.806174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f391756-477f-41b5-a1bc-1f2de8d834a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:08.806459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f391756-477f-41b5-a1bc-1f2de8d834a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:08.807425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f391756-477f-41b5-a1bc-1f2de8d834a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:08.808327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f391756-477f-41b5-a1bc-1f2de8d834a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:08.810400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f391756-477f-41b5-a1bc-1f2de8d834a8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:08.845444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 941/1855] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:08.850107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c3d6-75df-4f0e-8288-fab577d6bb98 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:08.885317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c3d6-75df-4f0e-8288-fab577d6bb98 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:08.892374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:08.893307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:08.893913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:08.894411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] PATH_INFO: `/v3/projects/abecf29ba1b64db68cb93a6d817635e5/users/781229479c0e41e39f5bacf2f6db07d9/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:08.898831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abecf29ba1b64db68cb93a6d817635e5, user_id=781229479c0e41e39f5bacf2f6db07d9, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:08.902827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:08.926801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 915/1856] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:08 2022] PUT /identity/v3/projects/abecf29ba1b64db68cb93a6d817635e5/users/781229479c0e41e39f5bacf2f6db07d9/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:08.941308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19176-fb6a-4e88-935b-aab8722b3e27 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:08.941673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19176-fb6a-4e88-935b-aab8722b3e27 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:08.942063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19176-fb6a-4e88-935b-aab8722b3e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:08.996109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d6a19176-fb6a-4e88-935b-aab8722b3e27 None None] MFA Rules not processed for user `781229479c0e41e39f5bacf2f6db07d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:08.998267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d6a19176-fb6a-4e88-935b-aab8722b3e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:09.133220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:09.204671 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 942/1857] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:09.217883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:09.221501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:09.446344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e25bf-fcc3-43cc-8d00-31b6a1f38756 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:09.464192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6024b-02c7-4ed3-b244-311a292b9e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:09.464922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6024b-02c7-4ed3-b244-311a292b9e0d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:09.465208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6024b-02c7-4ed3-b244-311a292b9e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:09.465571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6024b-02c7-4ed3-b244-311a292b9e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:09.506487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6024b-02c7-4ed3-b244-311a292b9e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:09.508550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6024b-02c7-4ed3-b244-311a292b9e0d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:09.553572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 916/1858] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:09.562814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a19176-fb6a-4e88-935b-aab8722b3e27 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:09.564267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d6a19176-fb6a-4e88-935b-aab8722b3e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:09.834422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a19176-fb6a-4e88-935b-aab8722b3e27 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:09.856978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:09.859181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:09.860346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:09.860810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] PATH_INFO: `/v3/projects/bb02136e4dc64626adcef53a0c44fab9` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:09.864277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb02136e4dc64626adcef53a0c44fab9)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:09.871426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:10.034001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb02136e4dc64626adcef53a0c44fab9', 'request_id': 'req-38be07bf-5b2d-4b99-bf46-822192a8fc8f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.062869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb02136e4dc64626adcef53a0c44fab9', 'request_id': 'req-38be07bf-5b2d-4b99-bf46-822192a8fc8f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.092570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb02136e4dc64626adcef53a0c44fab9', 'request_id': 'req-38be07bf-5b2d-4b99-bf46-822192a8fc8f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.143342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] The token cache is being invalidate because project bb02136e4dc64626adcef53a0c44fab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:10.146024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.149182 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-38be07bf-5b2d-4b99-bf46-822192a8fc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.163897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 943/1859] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:09 2022] DELETE /identity/v3/projects/bb02136e4dc64626adcef53a0c44fab9 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:10.164619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6024b-02c7-4ed3-b244-311a292b9e0d service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:10.185728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6024b-02c7-4ed3-b244-311a292b9e0d service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:10.331521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:10.332318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:10.332658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:10.333038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] PATH_INFO: `/v3/users/6e2096e9095449748d31a3765a97a7cc` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:10.336169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2096e9095449748d31a3765a97a7cc)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:10.338975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:10.409900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:10.440375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2096e9095449748d31a3765a97a7cc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.464714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2096e9095449748d31a3765a97a7cc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.465728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2096e9095449748d31a3765a97a7cc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.480598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2096e9095449748d31a3765a97a7cc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.531464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2096e9095449748d31a3765a97a7cc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:10.544223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 917/1860] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:10 2022] DELETE /identity/v3/users/6e2096e9095449748d31a3765a97a7cc => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:12.333045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd4ac-d96b-42bc-abda-ee02973f732d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:12.333554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd4ac-d96b-42bc-abda-ee02973f732d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:12.333805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd4ac-d96b-42bc-abda-ee02973f732d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:12.337174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 944/1861] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:38:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:38:12.345283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:12.346854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:12.467481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bd4ac-d96b-42bc-abda-ee02973f732d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:12.469740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a95bd4ac-d96b-42bc-abda-ee02973f732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:12.535180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fb953-77e0-4e6c-b031-c5ba21ae149c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:12.543355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:12.544353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:12.544790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:12.545215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] PATH_INFO: `/v3/projects/9779bfaefa1140bab7db560eb45f0019` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:12.546579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9779bfaefa1140bab7db560eb45f0019)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:12.548661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:12.645368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bd4ac-d96b-42bc-abda-ee02973f732d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:12.645628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9779bfaefa1140bab7db560eb45f0019', 'request_id': 'req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.667583 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9779bfaefa1140bab7db560eb45f0019', 'request_id': 'req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.676122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:12.691486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9779bfaefa1140bab7db560eb45f0019', 'request_id': 'req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.730088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:12.730621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] The token cache is being invalidate because project 9779bfaefa1140bab7db560eb45f0019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:12.730968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.731081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:12.731433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:12.732033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] PATH_INFO: `/v3/projects/ba760e2eb0f54dcc9c7e05a0c63e42ec` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:12.732478 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c08c0fc-837c-44ed-a8d4-2f17dcfa8135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.733646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba760e2eb0f54dcc9c7e05a0c63e42ec)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:12.735862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:12.737683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 918/1862] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:12 2022] DELETE /identity/v3/projects/9779bfaefa1140bab7db560eb45f0019 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:12.738993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310af4-5519-4626-8b96-fdf1da0df015 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:12.739279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310af4-5519-4626-8b96-fdf1da0df015 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:12.739557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310af4-5519-4626-8b96-fdf1da0df015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:12.780334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-30310af4-5519-4626-8b96-fdf1da0df015 None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:12.782718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-30310af4-5519-4626-8b96-fdf1da0df015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:12.814753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba760e2eb0f54dcc9c7e05a0c63e42ec', 'request_id': 'req-4f7bdcce-eeac-4d17-b87e-de619297089e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.823504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba760e2eb0f54dcc9c7e05a0c63e42ec', 'request_id': 'req-4f7bdcce-eeac-4d17-b87e-de619297089e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.838913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba760e2eb0f54dcc9c7e05a0c63e42ec', 'request_id': 'req-4f7bdcce-eeac-4d17-b87e-de619297089e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.869703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:12.876279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] The token cache is being invalidate because project ba760e2eb0f54dcc9c7e05a0c63e42ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:12.876592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.877497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:12.885395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 945/1863] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:12 2022] DELETE /identity/v3/projects/ba760e2eb0f54dcc9c7e05a0c63e42ec => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:12.886390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:12.892790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:12.946858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 919/1864] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:38:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:12.947948 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30310af4-5519-4626-8b96-fdf1da0df015 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:12.950074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-30310af4-5519-4626-8b96-fdf1da0df015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:12.971962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30310af4-5519-4626-8b96-fdf1da0df015 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:12.983011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bdcce-eeac-4d17-b87e-de619297089e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:12.983286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:12.993004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:12.994024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:12.994433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] PATH_INFO: `/v3/users/57e2801ab59f4f07acec902d3cfa5f13` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:12.995915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e2801ab59f4f07acec902d3cfa5f13)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:12.998033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:12.998487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.000178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.000648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.001765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] PATH_INFO: `/v3/users/9e49bf6589144816add2e325c2cf9fd7` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.007351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e49bf6589144816add2e325c2cf9fd7)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.010380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.076427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e2801ab59f4f07acec902d3cfa5f13'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.092605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e2801ab59f4f07acec902d3cfa5f13'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.095801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e2801ab59f4f07acec902d3cfa5f13'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.106503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e49bf6589144816add2e325c2cf9fd7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.114660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e2801ab59f4f07acec902d3cfa5f13'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.118944 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e49bf6589144816add2e325c2cf9fd7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.120753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e49bf6589144816add2e325c2cf9fd7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.124214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:13.141655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-901959ff-c7bd-43f1-9501-7e37a0c76cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e2801ab59f4f07acec902d3cfa5f13'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.152186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e49bf6589144816add2e325c2cf9fd7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.154171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e49bf6589144816add2e325c2cf9fd7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:13.167670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 920/1865] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:12 2022] DELETE /identity/v3/users/57e2801ab59f4f07acec902d3cfa5f13 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:13.169388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14132809-2728-4b39-a332-7af2c5e7d27f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.169721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14132809-2728-4b39-a332-7af2c5e7d27f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.169991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14132809-2728-4b39-a332-7af2c5e7d27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.195054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-14132809-2728-4b39-a332-7af2c5e7d27f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:13.196477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-14132809-2728-4b39-a332-7af2c5e7d27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:13.203584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 946/1866] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:12 2022] DELETE /identity/v3/users/9e49bf6589144816add2e325c2cf9fd7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:13.205498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:13.207647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:13.299039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 921/1867] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:13.301806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aef97a-414c-4999-8fb8-9dcc845343dc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.302107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aef97a-414c-4999-8fb8-9dcc845343dc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.302382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aef97a-414c-4999-8fb8-9dcc845343dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.333355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-80aef97a-414c-4999-8fb8-9dcc845343dc None None] MFA Rules not processed for user `289c7d8b89cd445a924f5ac07e85fe48`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:13.334738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-80aef97a-414c-4999-8fb8-9dcc845343dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:13.354320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99bf02-0f87-4c7e-8152-ef9e38430841 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:13.365560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781be02-3c39-4ffd-9e66-6fb619b414fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.366339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781be02-3c39-4ffd-9e66-6fb619b414fb service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.366620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781be02-3c39-4ffd-9e66-6fb619b414fb service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.367205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781be02-3c39-4ffd-9e66-6fb619b414fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.369507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5781be02-3c39-4ffd-9e66-6fb619b414fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:13.383043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:13.393549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781be02-3c39-4ffd-9e66-6fb619b414fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.397813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781be02-3c39-4ffd-9e66-6fb619b414fb service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.406909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 922/1868] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:13.408640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9881c-ca6e-4f6c-902e-d424435cc192 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.409116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9881c-ca6e-4f6c-902e-d424435cc192 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.409645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9881c-ca6e-4f6c-902e-d424435cc192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.422785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 947/1869] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:13.424258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426bc6e-300c-4690-9c3c-9847d357d4fc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.424646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426bc6e-300c-4690-9c3c-9847d357d4fc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.425025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426bc6e-300c-4690-9c3c-9847d357d4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.437906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-24f9881c-ca6e-4f6c-902e-d424435cc192 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:13.439319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-24f9881c-ca6e-4f6c-902e-d424435cc192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:13.449760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 923/1870] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:13.454772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1426bc6e-300c-4690-9c3c-9847d357d4fc None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:13.456314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1426bc6e-300c-4690-9c3c-9847d357d4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:13.468608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 948/1871] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:13.472500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9881c-ca6e-4f6c-902e-d424435cc192 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:13.486614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426bc6e-300c-4690-9c3c-9847d357d4fc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:13.488360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1426bc6e-300c-4690-9c3c-9847d357d4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:13.488610 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9881c-ca6e-4f6c-902e-d424435cc192 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:13.495603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.496141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.496654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.497377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.497747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:13.498066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.500573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.517286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 924/1872] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:38:13.521499 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:13.542867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b034aa-1096-4b7f-a551-3c1391823f5f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:13.551947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f5570-b38f-4484-a6c7-81b4a9ecd935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.552949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f5570-b38f-4484-a6c7-81b4a9ecd935 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.553384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f5570-b38f-4484-a6c7-81b4a9ecd935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.553762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f5570-b38f-4484-a6c7-81b4a9ecd935 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.554925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f5570-b38f-4484-a6c7-81b4a9ecd935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.557973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f5570-b38f-4484-a6c7-81b4a9ecd935 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.584514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426bc6e-300c-4690-9c3c-9847d357d4fc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:13.589140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4916b9-a662-450f-a070-eef74c2673db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.590027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4916b9-a662-450f-a070-eef74c2673db service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.590569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4916b9-a662-450f-a070-eef74c2673db service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.590880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4916b9-a662-450f-a070-eef74c2673db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.603472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 925/1873] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:13 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:38:13.608224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f5570-b38f-4484-a6c7-81b4a9ecd935 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:13.610233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4916b9-a662-450f-a070-eef74c2673db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.612555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4916b9-a662-450f-a070-eef74c2673db service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.630243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 949/1874] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:13.718217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f5570-b38f-4484-a6c7-81b4a9ecd935 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:13.723557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861daf1-d64f-46ec-a306-6d8df72effdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.724338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861daf1-d64f-46ec-a306-6d8df72effdc admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.724716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861daf1-d64f-46ec-a306-6d8df72effdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.725059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861daf1-d64f-46ec-a306-6d8df72effdc admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.725884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861daf1-d64f-46ec-a306-6d8df72effdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.728860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861daf1-d64f-46ec-a306-6d8df72effdc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.755898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 926/1875] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:13 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:38:13.760389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4916b9-a662-450f-a070-eef74c2673db service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:13.775376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4916b9-a662-450f-a070-eef74c2673db service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:13.782553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267adc4-c6ec-45de-915e-b1992d007177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.783192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267adc4-c6ec-45de-915e-b1992d007177 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.783456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267adc4-c6ec-45de-915e-b1992d007177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.783774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267adc4-c6ec-45de-915e-b1992d007177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.784647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267adc4-c6ec-45de-915e-b1992d007177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.786474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267adc4-c6ec-45de-915e-b1992d007177 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.799386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 950/1876] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:13.803797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861daf1-d64f-46ec-a306-6d8df72effdc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:13.814942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861daf1-d64f-46ec-a306-6d8df72effdc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:13.819836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22303e70-a88d-456d-9dba-78a585565dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.820469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22303e70-a88d-456d-9dba-78a585565dab admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.820814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22303e70-a88d-456d-9dba-78a585565dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.821092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22303e70-a88d-456d-9dba-78a585565dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.821814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22303e70-a88d-456d-9dba-78a585565dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.823475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22303e70-a88d-456d-9dba-78a585565dab admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.835338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 927/1877] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:13.839192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267adc4-c6ec-45de-915e-b1992d007177 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:13.851996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267adc4-c6ec-45de-915e-b1992d007177 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:13.858867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:13.858867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.858867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.858867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] PATH_INFO: `/v3/projects/b52f008781f64d09b706d478c8fe8922/users/411ff3f0306848ee91108343982806ca/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.878202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52f008781f64d09b706d478c8fe8922, user_id=411ff3f0306848ee91108343982806ca, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:13.880486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:13.895101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 951/1878] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:13 2022] PUT /identity/v3/projects/b52f008781f64d09b706d478c8fe8922/users/411ff3f0306848ee91108343982806ca/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:13.901219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f498ee-cea2-48a3-9737-4332f8500e64 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:13.901672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f498ee-cea2-48a3-9737-4332f8500e64 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:13.901959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f498ee-cea2-48a3-9737-4332f8500e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:13.927535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-73f498ee-cea2-48a3-9737-4332f8500e64 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:13.928771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-73f498ee-cea2-48a3-9737-4332f8500e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:14.020624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 928/1879] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:14.082667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:14.085656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:14.116608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:14.166266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102ad5f-2455-49ea-92dc-6d093c2e3c53 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:14.174085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afff2e-ca31-4be0-a72f-f744dd74f37f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:14.174719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afff2e-ca31-4be0-a72f-f744dd74f37f service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:14.175037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afff2e-ca31-4be0-a72f-f744dd74f37f service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:14.175517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afff2e-ca31-4be0-a72f-f744dd74f37f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:14.194929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afff2e-ca31-4be0-a72f-f744dd74f37f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:14.197142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afff2e-ca31-4be0-a72f-f744dd74f37f service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:14.209369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 952/1880] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:14.611371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f498ee-cea2-48a3-9737-4332f8500e64 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:14.614083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-73f498ee-cea2-48a3-9737-4332f8500e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:14.627730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f498ee-cea2-48a3-9737-4332f8500e64 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:14.635099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:14.635932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:14.636376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:14.637067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] PATH_INFO: `/v3/projects/970e12eb04c042018f72bc5d8b8abc55` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:14.638619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] RBAC: Authorizing `identity:delete_project(project_id=970e12eb04c042018f72bc5d8b8abc55)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:14.640802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:14.688423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:14.730371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '970e12eb04c042018f72bc5d8b8abc55', 'request_id': 'req-74da5699-0860-474a-85e5-29b832c7536f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:14.744569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '970e12eb04c042018f72bc5d8b8abc55', 'request_id': 'req-74da5699-0860-474a-85e5-29b832c7536f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:14.778645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '970e12eb04c042018f72bc5d8b8abc55', 'request_id': 'req-74da5699-0860-474a-85e5-29b832c7536f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:14.805771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] The token cache is being invalidate because project 970e12eb04c042018f72bc5d8b8abc55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:14.806823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:14.808156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:14.810413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 929/1881] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:14 2022] DELETE /identity/v3/projects/970e12eb04c042018f72bc5d8b8abc55 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:14.814605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afff2e-ca31-4be0-a72f-f744dd74f37f service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:14.838316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afff2e-ca31-4be0-a72f-f744dd74f37f service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:14.964483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:14.966995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:14.967290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:14.967965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] PATH_INFO: `/v3/users/1f6d3c7ed5c94f579cba6c77d762641e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:14.969910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6d3c7ed5c94f579cba6c77d762641e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:14.974044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:15.046650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:15.048906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:15.083315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5699-0860-474a-85e5-29b832c7536f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:15.098281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:15.099823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:15.101565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:15.101925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] PATH_INFO: `/v3/projects/b0361ee3aa6d46febd2aa21f94188e5a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:15.104785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0361ee3aa6d46febd2aa21f94188e5a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:15.107574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6d3c7ed5c94f579cba6c77d762641e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.112962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:15.135215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6d3c7ed5c94f579cba6c77d762641e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.135943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6d3c7ed5c94f579cba6c77d762641e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.185982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6d3c7ed5c94f579cba6c77d762641e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.189414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6d3c7ed5c94f579cba6c77d762641e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.230642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 953/1882] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:14 2022] DELETE /identity/v3/users/1f6d3c7ed5c94f579cba6c77d762641e => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:15.262173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0361ee3aa6d46febd2aa21f94188e5a', 'request_id': 'req-968cd134-ed49-4a01-9a2b-9a0891865c4e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.279172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0361ee3aa6d46febd2aa21f94188e5a', 'request_id': 'req-968cd134-ed49-4a01-9a2b-9a0891865c4e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.289704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0361ee3aa6d46febd2aa21f94188e5a', 'request_id': 'req-968cd134-ed49-4a01-9a2b-9a0891865c4e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.311194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] The token cache is being invalidate because project b0361ee3aa6d46febd2aa21f94188e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:15.311590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.312276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.314270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 930/1883] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:15 2022] DELETE /identity/v3/projects/b0361ee3aa6d46febd2aa21f94188e5a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:15.321910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:15.323380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:15.392228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:15.421568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750f297-4460-4477-a34c-1d9186d12ad7 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:15.511685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:15.513380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:15.514717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:15.516629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] PATH_INFO: `/v3/users/781229479c0e41e39f5bacf2f6db07d9` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:15.519194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=781229479c0e41e39f5bacf2f6db07d9)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:15.521623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:15.611704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '781229479c0e41e39f5bacf2f6db07d9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.623208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '781229479c0e41e39f5bacf2f6db07d9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.624156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '781229479c0e41e39f5bacf2f6db07d9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.647068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '781229479c0e41e39f5bacf2f6db07d9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.648620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8abbb4e-b693-416a-8781-86ed1d5904cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '781229479c0e41e39f5bacf2f6db07d9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:15.666914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 954/1884] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:15 2022] DELETE /identity/v3/users/781229479c0e41e39f5bacf2f6db07d9 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:16.521117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:16.522702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:16.536088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd134-ed49-4a01-9a2b-9a0891865c4e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:16.606614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:16.607341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:16.608187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:16.608551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] PATH_INFO: `/v3/projects/abecf29ba1b64db68cb93a6d817635e5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:16.609818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abecf29ba1b64db68cb93a6d817635e5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:16.611800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:16.655666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:16.672531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abecf29ba1b64db68cb93a6d817635e5', 'request_id': 'req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:16.680727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abecf29ba1b64db68cb93a6d817635e5', 'request_id': 'req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:16.688098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abecf29ba1b64db68cb93a6d817635e5', 'request_id': 'req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:16.705257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] The token cache is being invalidate because project abecf29ba1b64db68cb93a6d817635e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:16.705469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:16.706246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-025e0d2c-a74e-4d43-8a4a-bb88ccfc7245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:16.708068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 931/1885] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:16 2022] DELETE /identity/v3/projects/abecf29ba1b64db68cb93a6d817635e5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:16.712909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a588c1-fd43-428d-ae78-2858777b518b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:16.713430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a588c1-fd43-428d-ae78-2858777b518b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:16.713746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a588c1-fd43-428d-ae78-2858777b518b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:16.738138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f5a588c1-fd43-428d-ae78-2858777b518b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:16.739451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f5a588c1-fd43-428d-ae78-2858777b518b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:16.819710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 955/1886] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:16.828050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c79e3-c0af-4fb3-b144-9cf440b00b85 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:16.828449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c79e3-c0af-4fb3-b144-9cf440b00b85 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:16.829301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c79e3-c0af-4fb3-b144-9cf440b00b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:16.875892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4b5c79e3-c0af-4fb3-b144-9cf440b00b85 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:16.881289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4b5c79e3-c0af-4fb3-b144-9cf440b00b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:16.902745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 932/1887] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:16.904044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a588c1-fd43-428d-ae78-2858777b518b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:16.915547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a588c1-fd43-428d-ae78-2858777b518b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:16.925970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:16.926334 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:16.926437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c79e3-c0af-4fb3-b144-9cf440b00b85 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:16.926613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:16.926939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:16.927595 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:16.927896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:16.929816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:16.930457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4b5c79e3-c0af-4fb3-b144-9cf440b00b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:16.933732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:16.946239 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c79e3-c0af-4fb3-b144-9cf440b00b85 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:16.951869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 956/1888] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:38:16.953522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:16.954459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:16.955721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:16.956035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:16.956536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] PATH_INFO: `/v3/projects/499c252e7b614000a08aca9b2adfd27f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:16.958779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=499c252e7b614000a08aca9b2adfd27f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:16.961571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:16.969870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd5c4c-58f2-416b-b12c-ed1a7377806b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:16.978218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ebc46-2ecc-4135-ad1d-1c570390bb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:16.979255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ebc46-2ecc-4135-ad1d-1c570390bb5a admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:16.979896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ebc46-2ecc-4135-ad1d-1c570390bb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:16.980441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ebc46-2ecc-4135-ad1d-1c570390bb5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:16.981684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ebc46-2ecc-4135-ad1d-1c570390bb5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:16.986565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ebc46-2ecc-4135-ad1d-1c570390bb5a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.031151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499c252e7b614000a08aca9b2adfd27f', 'request_id': 'req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:17.043637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 957/1889] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:16 2022] POST /identity/v3/projects => generated 387 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:17.048524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499c252e7b614000a08aca9b2adfd27f', 'request_id': 'req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:17.050664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ebc46-2ecc-4135-ad1d-1c570390bb5a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:17.065556 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499c252e7b614000a08aca9b2adfd27f', 'request_id': 'req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:17.098965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] The token cache is being invalidate because project 499c252e7b614000a08aca9b2adfd27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:17.099420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:17.100282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-891d3799-76fe-4f2d-8d6c-c012daf9fb1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:17.112553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 933/1890] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:16 2022] DELETE /identity/v3/projects/499c252e7b614000a08aca9b2adfd27f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:17.115854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c948e-b922-457c-b3b8-1f5df1784e95 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.116412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c948e-b922-457c-b3b8-1f5df1784e95 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.116918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c948e-b922-457c-b3b8-1f5df1784e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.165138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ebc46-2ecc-4135-ad1d-1c570390bb5a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:17.171998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:17.172811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.173091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.173581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.174212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:17.176673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.183565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b85c948e-b922-457c-b3b8-1f5df1784e95 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:17.190520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b85c948e-b922-457c-b3b8-1f5df1784e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:17.216597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 934/1891] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:17.218243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707fa1f-6a49-44e9-ad6a-8e1aaa55dda4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.218634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707fa1f-6a49-44e9-ad6a-8e1aaa55dda4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.219022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707fa1f-6a49-44e9-ad6a-8e1aaa55dda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.229269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 958/1892] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:17 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:17.229269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:17.233086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:17.267860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0707fa1f-6a49-44e9-ad6a-8e1aaa55dda4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:17.270357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0707fa1f-6a49-44e9-ad6a-8e1aaa55dda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:17.283646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d50d8-855f-460d-baf1-60b9fc0a22bc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:17.300862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476c32-9b86-47d4-b9a8-95c9c018388a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:17.301783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476c32-9b86-47d4-b9a8-95c9c018388a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.304197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476c32-9b86-47d4-b9a8-95c9c018388a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.305504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476c32-9b86-47d4-b9a8-95c9c018388a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.307563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a476c32-9b86-47d4-b9a8-95c9c018388a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:17.309490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a476c32-9b86-47d4-b9a8-95c9c018388a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.315935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 935/1893] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:17.324340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707fa1f-6a49-44e9-ad6a-8e1aaa55dda4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:17.341113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707fa1f-6a49-44e9-ad6a-8e1aaa55dda4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:17.354099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 959/1894] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:17.360359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476c32-9b86-47d4-b9a8-95c9c018388a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:17.371924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:17.374905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.374905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.374905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.374905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:17.374905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:17.376203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.406612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476c32-9b86-47d4-b9a8-95c9c018388a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:17.416844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 936/1895] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:38:17.421068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:17.422339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bf815-8ad3-4421-aca4-74f66feed4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:17.422976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf815-8ad3-4421-aca4-74f66feed4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.424487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf815-8ad3-4421-aca4-74f66feed4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.424808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf815-8ad3-4421-aca4-74f66feed4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.427712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bf815-8ad3-4421-aca4-74f66feed4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:17.433543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bf815-8ad3-4421-aca4-74f66feed4dc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.438185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12822f6-a3ff-4440-a530-28ba84eeae35 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:17.447728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69e9c4-59f6-4b50-8b72-4c80d63461f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:17.448803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69e9c4-59f6-4b50-8b72-4c80d63461f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.449171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69e9c4-59f6-4b50-8b72-4c80d63461f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.449566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69e9c4-59f6-4b50-8b72-4c80d63461f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.450306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69e9c4-59f6-4b50-8b72-4c80d63461f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:17.452341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69e9c4-59f6-4b50-8b72-4c80d63461f3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.460805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 960/1896] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:17.464645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bf815-8ad3-4421-aca4-74f66feed4dc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:17.480885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bf815-8ad3-4421-aca4-74f66feed4dc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:17.493019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf15a57-3453-4690-b8a9-00af05cd8e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:17.494184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf15a57-3453-4690-b8a9-00af05cd8e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.494792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf15a57-3453-4690-b8a9-00af05cd8e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.495360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf15a57-3453-4690-b8a9-00af05cd8e4e admin admin] PATH_INFO: `/v3/projects/14258e6c31344b31aba242a1fb534c08/users/a913c7bce14f48ba9d113fefbdd7ae1a/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.517388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 937/1897] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:17 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:17.521188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69e9c4-59f6-4b50-8b72-4c80d63461f3 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:17.538081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf15a57-3453-4690-b8a9-00af05cd8e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=14258e6c31344b31aba242a1fb534c08, user_id=a913c7bce14f48ba9d113fefbdd7ae1a, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:17.544640 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf15a57-3453-4690-b8a9-00af05cd8e4e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.574931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 961/1898] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:17 2022] PUT /identity/v3/projects/14258e6c31344b31aba242a1fb534c08/users/a913c7bce14f48ba9d113fefbdd7ae1a/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:17.587303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36349b9-086e-420f-938f-a9ae982a91f7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.587587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36349b9-086e-420f-938f-a9ae982a91f7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.587862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36349b9-086e-420f-938f-a9ae982a91f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.643147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:17.659641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b36349b9-086e-420f-938f-a9ae982a91f7 None None] MFA Rules not processed for user `a913c7bce14f48ba9d113fefbdd7ae1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:17.661064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b36349b9-086e-420f-938f-a9ae982a91f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:17.680430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69e9c4-59f6-4b50-8b72-4c80d63461f3 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:17.802825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b232c55-01f5-4826-8c73-4db191f43237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:17.803690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232c55-01f5-4826-8c73-4db191f43237 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.804587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232c55-01f5-4826-8c73-4db191f43237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.804967 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b232c55-01f5-4826-8c73-4db191f43237 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.805734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b232c55-01f5-4826-8c73-4db191f43237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:17.822247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b232c55-01f5-4826-8c73-4db191f43237 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.845902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 962/1899] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:17.847887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878bce4-8376-4124-a77c-6531aa3132d7 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.848171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878bce4-8376-4124-a77c-6531aa3132d7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.848508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878bce4-8376-4124-a77c-6531aa3132d7 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.860244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 963/1900] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:38:17 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:38:17.863335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878bce4-8376-4124-a77c-6531aa3132d7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:17.875480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 938/1901] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:17 2022] POST /identity/v3/users => generated 294 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:38:17.876974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c774d7-d8e0-4c43-a1af-3ae8e18e8efc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.877352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c774d7-d8e0-4c43-a1af-3ae8e18e8efc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.877754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c774d7-d8e0-4c43-a1af-3ae8e18e8efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.917470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e6c774d7-d8e0-4c43-a1af-3ae8e18e8efc None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:17.919984 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e6c774d7-d8e0-4c43-a1af-3ae8e18e8efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:17.923679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878bce4-8376-4124-a77c-6531aa3132d7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:17.954452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ce6f3-c90f-42d6-b294-8964a8c2ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:17.955090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ce6f3-c90f-42d6-b294-8964a8c2ea0c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:17.955364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ce6f3-c90f-42d6-b294-8964a8c2ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:17.955629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ce6f3-c90f-42d6-b294-8964a8c2ea0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:17.959225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ce6f3-c90f-42d6-b294-8964a8c2ea0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:17.966537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ce6f3-c90f-42d6-b294-8964a8c2ea0c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:17.983697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:18.059485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 964/1902] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:17 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:18.061308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ce6f3-c90f-42d6-b294-8964a8c2ea0c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.124553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 939/1903] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:18.126701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c774d7-d8e0-4c43-a1af-3ae8e18e8efc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.158386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c774d7-d8e0-4c43-a1af-3ae8e18e8efc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:18.248930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ce6f3-c90f-42d6-b294-8964a8c2ea0c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:18.264593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3b3a4-ea5b-4138-9cb2-6d09b2fb4dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:18.267004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3b3a4-ea5b-4138-9cb2-6d09b2fb4dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.267584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3b3a4-ea5b-4138-9cb2-6d09b2fb4dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.267857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3b3a4-ea5b-4138-9cb2-6d09b2fb4dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.271177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3b3a4-ea5b-4138-9cb2-6d09b2fb4dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:18.276500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3b3a4-ea5b-4138-9cb2-6d09b2fb4dc2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:18.310209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 965/1904] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:18.313352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3b3a4-ea5b-4138-9cb2-6d09b2fb4dc2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.351418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb91c6-8b52-44a2-8696-37001fc35237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:18.352048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb91c6-8b52-44a2-8696-37001fc35237 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.352365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb91c6-8b52-44a2-8696-37001fc35237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.352588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb91c6-8b52-44a2-8696-37001fc35237 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.357024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3b3a4-ea5b-4138-9cb2-6d09b2fb4dc2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:18.358432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb91c6-8b52-44a2-8696-37001fc35237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:18.360839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb91c6-8b52-44a2-8696-37001fc35237 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:18.369673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad904c-468b-4034-ac16-cb4573feed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:18.370059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad904c-468b-4034-ac16-cb4573feed16 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.371738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad904c-468b-4034-ac16-cb4573feed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.372018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad904c-468b-4034-ac16-cb4573feed16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.372670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad904c-468b-4034-ac16-cb4573feed16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:18.374475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad904c-468b-4034-ac16-cb4573feed16 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:18.439071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 966/1905] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:18.444211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad904c-468b-4034-ac16-cb4573feed16 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.465656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad904c-468b-4034-ac16-cb4573feed16 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:18.478218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 940/1906] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:18 2022] POST /identity/v3/users => generated 300 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:18.479232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775febdf-fa07-43f3-9c20-b3696173c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:18.479921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775febdf-fa07-43f3-9c20-b3696173c67a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.480197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775febdf-fa07-43f3-9c20-b3696173c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.480659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775febdf-fa07-43f3-9c20-b3696173c67a admin admin] PATH_INFO: `/v3/projects/e8e19d8d4d9441b8a4394107c6f0bb1a/users/fb36de7572ff41968f3163c9be61971a/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.483765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb91c6-8b52-44a2-8696-37001fc35237 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.498244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb91c6-8b52-44a2-8696-37001fc35237 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:18.504638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775febdf-fa07-43f3-9c20-b3696173c67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e19d8d4d9441b8a4394107c6f0bb1a, user_id=fb36de7572ff41968f3163c9be61971a, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:18.509698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a03df5-493f-4aa0-a1b8-0683a76944bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:18.512482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a03df5-493f-4aa0-a1b8-0683a76944bb admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.520175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a03df5-493f-4aa0-a1b8-0683a76944bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.520717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775febdf-fa07-43f3-9c20-b3696173c67a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:18.525156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a03df5-493f-4aa0-a1b8-0683a76944bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.526814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a03df5-493f-4aa0-a1b8-0683a76944bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:18.529039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a03df5-493f-4aa0-a1b8-0683a76944bb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:18.532825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:18.554994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 967/1907] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:18 2022] PUT /identity/v3/projects/e8e19d8d4d9441b8a4394107c6f0bb1a/users/fb36de7572ff41968f3163c9be61971a/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:18.560782 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 941/1908] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:18.562377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775febdf-fa07-43f3-9c20-b3696173c67a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.574785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e887365-2813-4793-868f-4c9c9624accf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.575101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e887365-2813-4793-868f-4c9c9624accf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.575330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e887365-2813-4793-868f-4c9c9624accf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.650582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0e887365-2813-4793-868f-4c9c9624accf None None] MFA Rules not processed for user `fb36de7572ff41968f3163c9be61971a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:18.653337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0e887365-2813-4793-868f-4c9c9624accf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:18.700618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775febdf-fa07-43f3-9c20-b3696173c67a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:18.719601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c91fe-347d-4a6c-9b11-685eaba56008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:18.720289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c91fe-347d-4a6c-9b11-685eaba56008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.726286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c91fe-347d-4a6c-9b11-685eaba56008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.726848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c91fe-347d-4a6c-9b11-685eaba56008 admin admin] PATH_INFO: `/v3/projects/02f7cb109a3e42dba40d69f71d352664/users/c1684b32b1424af48b054a13c22761eb/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.750130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c91fe-347d-4a6c-9b11-685eaba56008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f7cb109a3e42dba40d69f71d352664, user_id=c1684b32b1424af48b054a13c22761eb, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:18.753365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c91fe-347d-4a6c-9b11-685eaba56008 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:18.764527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 942/1909] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:18.772585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e887365-2813-4793-868f-4c9c9624accf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.781539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 968/1910] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:18 2022] PUT /identity/v3/projects/02f7cb109a3e42dba40d69f71d352664/users/c1684b32b1424af48b054a13c22761eb/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:18.786634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c91fe-347d-4a6c-9b11-685eaba56008 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.794371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e887365-2813-4793-868f-4c9c9624accf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:18.836285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:18.893503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c91fe-347d-4a6c-9b11-685eaba56008 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:18.900830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d6927-6a6d-4f37-88d9-381347071a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:18.902136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d6927-6a6d-4f37-88d9-381347071a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.902602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d6927-6a6d-4f37-88d9-381347071a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.902948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d6927-6a6d-4f37-88d9-381347071a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.904766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d6927-6a6d-4f37-88d9-381347071a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:18.908861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:18.909627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:18.909912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:18.910186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:18.910820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:18.912748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:18.933537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d6927-6a6d-4f37-88d9-381347071a73 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:18.977457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 943/1911] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:18 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:18.981650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:18.984676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:18.990973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 969/1912] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:18.990973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d6927-6a6d-4f37-88d9-381347071a73 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:19.006354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d6927-6a6d-4f37-88d9-381347071a73 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.119966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56711f8c-0a00-4107-b635-9153b2d3cb52 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.122274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee87949-c627-4524-ac8c-21cf7af054c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.122901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee87949-c627-4524-ac8c-21cf7af054c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.123181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee87949-c627-4524-ac8c-21cf7af054c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.123455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee87949-c627-4524-ac8c-21cf7af054c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.124085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee87949-c627-4524-ac8c-21cf7af054c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.126123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee87949-c627-4524-ac8c-21cf7af054c9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.138439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd5f0f-07bc-4827-b2e5-ff45ba425c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.139670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd5f0f-07bc-4827-b2e5-ff45ba425c53 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.140171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd5f0f-07bc-4827-b2e5-ff45ba425c53 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.140544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd5f0f-07bc-4827-b2e5-ff45ba425c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.149778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:19.156466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 970/1913] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:18 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:19.156466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee87949-c627-4524-ac8c-21cf7af054c9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:19.157539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd5f0f-07bc-4827-b2e5-ff45ba425c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.159526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd5f0f-07bc-4827-b2e5-ff45ba425c53 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.169687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee87949-c627-4524-ac8c-21cf7af054c9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.177833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c710c-7a4f-4ea8-937a-8c1abea65d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.178656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c710c-7a4f-4ea8-937a-8c1abea65d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.179015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c710c-7a4f-4ea8-937a-8c1abea65d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.179677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c710c-7a4f-4ea8-937a-8c1abea65d2a admin admin] PATH_INFO: `/v3/projects/02f7cb109a3e42dba40d69f71d352664/users/c1684b32b1424af48b054a13c22761eb/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.185617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c710c-7a4f-4ea8-937a-8c1abea65d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f7cb109a3e42dba40d69f71d352664, user_id=c1684b32b1424af48b054a13c22761eb, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.187851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 944/1914] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:19.189006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd5f0f-07bc-4827-b2e5-ff45ba425c53 service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:19.193610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c710c-7a4f-4ea8-937a-8c1abea65d2a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.208202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd5f0f-07bc-4827-b2e5-ff45ba425c53 service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.223043 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 971/1915] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:19 2022] PUT /identity/v3/projects/02f7cb109a3e42dba40d69f71d352664/users/c1684b32b1424af48b054a13c22761eb/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:19.235035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5aba81-c08d-45e5-821a-9fabc3a2fc17 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.235318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5aba81-c08d-45e5-821a-9fabc3a2fc17 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.235638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5aba81-c08d-45e5-821a-9fabc3a2fc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.303892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0d5aba81-c08d-45e5-821a-9fabc3a2fc17 None None] MFA Rules not processed for user `c1684b32b1424af48b054a13c22761eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:19.307844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0d5aba81-c08d-45e5-821a-9fabc3a2fc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:19.334742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c58974-df8d-4266-9506-9c8f66a875f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.336017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c58974-df8d-4266-9506-9c8f66a875f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.337850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c58974-df8d-4266-9506-9c8f66a875f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.338591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c58974-df8d-4266-9506-9c8f66a875f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.339905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c58974-df8d-4266-9506-9c8f66a875f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.344295 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c58974-df8d-4266-9506-9c8f66a875f5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.381245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 945/1916] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:19.382407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c58974-df8d-4266-9506-9c8f66a875f5 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:19.414066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c58974-df8d-4266-9506-9c8f66a875f5 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.430107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237c8f3-670d-4a6e-9bde-99b4d22d9daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.432823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c8f3-670d-4a6e-9bde-99b4d22d9daf admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.433366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c8f3-670d-4a6e-9bde-99b4d22d9daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.433871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237c8f3-670d-4a6e-9bde-99b4d22d9daf admin admin] PATH_INFO: `/v3/projects/c4713904e2df4aeeb64b7abcbddbbc9f/users/50c37354201b459a8d09fa8796085a1d/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.465132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237c8f3-670d-4a6e-9bde-99b4d22d9daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4713904e2df4aeeb64b7abcbddbbc9f, user_id=50c37354201b459a8d09fa8796085a1d, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.473224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237c8f3-670d-4a6e-9bde-99b4d22d9daf admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.493036 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 946/1917] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:19 2022] PUT /identity/v3/projects/c4713904e2df4aeeb64b7abcbddbbc9f/users/50c37354201b459a8d09fa8796085a1d/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:19.497175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237c8f3-670d-4a6e-9bde-99b4d22d9daf admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:19.505316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 972/1918] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:19.519132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5aba81-c08d-45e5-821a-9fabc3a2fc17 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:19.520572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0d5aba81-c08d-45e5-821a-9fabc3a2fc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:19.580278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237c8f3-670d-4a6e-9bde-99b4d22d9daf admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.586205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45649c95-7280-49ab-ba7f-b5bb13f1ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.586823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45649c95-7280-49ab-ba7f-b5bb13f1ac6b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.586823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45649c95-7280-49ab-ba7f-b5bb13f1ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.587204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45649c95-7280-49ab-ba7f-b5bb13f1ac6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.587993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45649c95-7280-49ab-ba7f-b5bb13f1ac6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.589764 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45649c95-7280-49ab-ba7f-b5bb13f1ac6b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.591103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:19.613014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 947/1919] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:38:19.615178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45649c95-7280-49ab-ba7f-b5bb13f1ac6b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:19.628522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45649c95-7280-49ab-ba7f-b5bb13f1ac6b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.635057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453e127-4739-4190-8275-bf0c92325a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.636022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453e127-4739-4190-8275-bf0c92325a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.636430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453e127-4739-4190-8275-bf0c92325a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.636804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453e127-4739-4190-8275-bf0c92325a81 admin admin] PATH_INFO: `/v3/projects/c4713904e2df4aeeb64b7abcbddbbc9f/users/50c37354201b459a8d09fa8796085a1d/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.638971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453e127-4739-4190-8275-bf0c92325a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4713904e2df4aeeb64b7abcbddbbc9f, user_id=50c37354201b459a8d09fa8796085a1d, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.643959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453e127-4739-4190-8275-bf0c92325a81 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.661496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5aba81-c08d-45e5-821a-9fabc3a2fc17 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.662395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 948/1920] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:19 2022] PUT /identity/v3/projects/c4713904e2df4aeeb64b7abcbddbbc9f/users/50c37354201b459a8d09fa8796085a1d/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:19.675373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f26e99-3dd5-4840-9d43-c362a8b52316 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.675816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f26e99-3dd5-4840-9d43-c362a8b52316 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.676186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f26e99-3dd5-4840-9d43-c362a8b52316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.716030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e0f26e99-3dd5-4840-9d43-c362a8b52316 None None] MFA Rules not processed for user `50c37354201b459a8d09fa8796085a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:19.717635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e0f26e99-3dd5-4840-9d43-c362a8b52316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:19.776698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:19.794960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262570-3638-49b8-a149-fea6f6e53b01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.795596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262570-3638-49b8-a149-fea6f6e53b01 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.795878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262570-3638-49b8-a149-fea6f6e53b01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.796147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262570-3638-49b8-a149-fea6f6e53b01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.824908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 949/1921] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:19.836411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f26e99-3dd5-4840-9d43-c362a8b52316 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:19.838337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e0f26e99-3dd5-4840-9d43-c362a8b52316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:19.865130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f26e99-3dd5-4840-9d43-c362a8b52316 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:19.875703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b6eb1-8a6c-4958-87d2-f05aa9b234c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:19.876765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6eb1-8a6c-4958-87d2-f05aa9b234c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:19.876878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6eb1-8a6c-4958-87d2-f05aa9b234c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:19.877135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6eb1-8a6c-4958-87d2-f05aa9b234c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:19.900732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b6eb1-8a6c-4958-87d2-f05aa9b234c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.902240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b6eb1-8a6c-4958-87d2-f05aa9b234c2 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.914439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 950/1922] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:19.935874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64262570-3638-49b8-a149-fea6f6e53b01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:19.939450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64262570-3638-49b8-a149-fea6f6e53b01 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:19.989257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 973/1923] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:20.124927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b6eb1-8a6c-4958-87d2-f05aa9b234c2 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:20.212022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262570-3638-49b8-a149-fea6f6e53b01 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:20.322771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b6eb1-8a6c-4958-87d2-f05aa9b234c2 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:20.335397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:20.337969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:20.338474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:20.339102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] PATH_INFO: `/v3/users/a913c7bce14f48ba9d113fefbdd7ae1a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:20.341912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a913c7bce14f48ba9d113fefbdd7ae1a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:20.348368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:20.362600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262570-3638-49b8-a149-fea6f6e53b01 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:20.376733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:20.378491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:20.378865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:20.379233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] PATH_INFO: `/v3/users/fb36de7572ff41968f3163c9be61971a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:20.382385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb36de7572ff41968f3163c9be61971a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:20.387961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:20.448110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a913c7bce14f48ba9d113fefbdd7ae1a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.466472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a913c7bce14f48ba9d113fefbdd7ae1a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.469156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a913c7bce14f48ba9d113fefbdd7ae1a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.474624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:20.489383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a913c7bce14f48ba9d113fefbdd7ae1a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.500058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb36de7572ff41968f3163c9be61971a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.516347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb36de7572ff41968f3163c9be61971a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.517513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb36de7572ff41968f3163c9be61971a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.523300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-fc07b067-164d-49bb-9b98-d766ef3f8c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a913c7bce14f48ba9d113fefbdd7ae1a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.540290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 951/1924] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:20 2022] DELETE /identity/v3/users/a913c7bce14f48ba9d113fefbdd7ae1a => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:20.545864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abdf50-1193-4718-a7ab-8fe0d5449c61 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:20.546344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abdf50-1193-4718-a7ab-8fe0d5449c61 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:20.547017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abdf50-1193-4718-a7ab-8fe0d5449c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:20.549097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb36de7572ff41968f3163c9be61971a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.551765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-31375890-ba64-472b-9187-e3173575e5ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb36de7572ff41968f3163c9be61971a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:20.583559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-45abdf50-1193-4718-a7ab-8fe0d5449c61 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:20.584109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 974/1925] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:20 2022] DELETE /identity/v3/users/fb36de7572ff41968f3163c9be61971a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:20.585076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-45abdf50-1193-4718-a7ab-8fe0d5449c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:20.589482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e69187-2dfe-47c8-9bed-d36c5825f594 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:20.590666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e69187-2dfe-47c8-9bed-d36c5825f594 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:20.591273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e69187-2dfe-47c8-9bed-d36c5825f594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:20.626272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b7e69187-2dfe-47c8-9bed-d36c5825f594 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:20.631737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b7e69187-2dfe-47c8-9bed-d36c5825f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:20.712493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:20.749389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 952/1926] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:20.766805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abdf50-1193-4718-a7ab-8fe0d5449c61 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:20.767361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 975/1927] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:20.768355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-45abdf50-1193-4718-a7ab-8fe0d5449c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:20.782736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e69187-2dfe-47c8-9bed-d36c5825f594 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:20.791598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b7e69187-2dfe-47c8-9bed-d36c5825f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:20.911907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abdf50-1193-4718-a7ab-8fe0d5449c61 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:20.918415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:20.919926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:20.920341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:20.920661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:20.922652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e69187-2dfe-47c8-9bed-d36c5825f594 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:20.933170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:20.933839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:20.934119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:20.934390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:20.936140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:20.943264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:20.959086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:20.960747 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:20.971019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 953/1928] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:20.981280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 976/1929] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:22.597706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:22.599169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:22.613337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66670f17-f8cc-4146-bd17-d24a91c6b7ba service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:22.621098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:22.621799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:22.622079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:22.622349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] PATH_INFO: `/v3/projects/14258e6c31344b31aba242a1fb534c08` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:22.623500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14258e6c31344b31aba242a1fb534c08)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:22.625807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:22.698427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:22.700266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:22.725046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee002b-1531-43a0-9c96-aad9d78b6de8 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:22.743780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14258e6c31344b31aba242a1fb534c08', 'request_id': 'req-1c742cc4-35f8-441c-83b4-657453c09766'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.757339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14258e6c31344b31aba242a1fb534c08', 'request_id': 'req-1c742cc4-35f8-441c-83b4-657453c09766'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.767949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14258e6c31344b31aba242a1fb534c08', 'request_id': 'req-1c742cc4-35f8-441c-83b4-657453c09766'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.793747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] The token cache is being invalidate because project 14258e6c31344b31aba242a1fb534c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:22.794163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.795396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.800303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 954/1930] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:22 2022] DELETE /identity/v3/projects/14258e6c31344b31aba242a1fb534c08 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:22.801191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:22.803071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:22.826093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:22.838956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:22.841287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:22.842247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:22.843010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] PATH_INFO: `/v3/projects/e8e19d8d4d9441b8a4394107c6f0bb1a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:22.845057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e19d8d4d9441b8a4394107c6f0bb1a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:22.848056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:22.892730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742cc4-35f8-441c-83b4-657453c09766 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:22.901252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:22.903019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:22.903231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:22.903509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] PATH_INFO: `/v3/users/c1684b32b1424af48b054a13c22761eb` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:22.905470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1684b32b1424af48b054a13c22761eb)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:22.908835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:22.916917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e19d8d4d9441b8a4394107c6f0bb1a', 'request_id': 'req-a8bcb7ca-8909-465a-8518-f0b61f7c503b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.926991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e19d8d4d9441b8a4394107c6f0bb1a', 'request_id': 'req-a8bcb7ca-8909-465a-8518-f0b61f7c503b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.941563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:22.942436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e19d8d4d9441b8a4394107c6f0bb1a', 'request_id': 'req-a8bcb7ca-8909-465a-8518-f0b61f7c503b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.969773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] The token cache is being invalidate because project e8e19d8d4d9441b8a4394107c6f0bb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:22.970898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.972156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:22.974593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 977/1931] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:22 2022] DELETE /identity/v3/projects/e8e19d8d4d9441b8a4394107c6f0bb1a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:22.978534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:22.980233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:23.018183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1684b32b1424af48b054a13c22761eb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.030247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1684b32b1424af48b054a13c22761eb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.031561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1684b32b1424af48b054a13c22761eb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.045702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1684b32b1424af48b054a13c22761eb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.063728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcb7ca-8909-465a-8518-f0b61f7c503b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:23.073490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:23.074147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:23.074425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:23.074696 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] PATH_INFO: `/v3/users/50c37354201b459a8d09fa8796085a1d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:23.077497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c37354201b459a8d09fa8796085a1d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:23.080989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:23.083012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1684b32b1424af48b054a13c22761eb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.093964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 955/1932] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:22 2022] DELETE /identity/v3/users/c1684b32b1424af48b054a13c22761eb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:23.154567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c37354201b459a8d09fa8796085a1d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.170322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c37354201b459a8d09fa8796085a1d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.174139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c37354201b459a8d09fa8796085a1d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.208013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c37354201b459a8d09fa8796085a1d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.209504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c37354201b459a8d09fa8796085a1d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:23.253203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 978/1933] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:22 2022] DELETE /identity/v3/users/50c37354201b459a8d09fa8796085a1d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:24.257587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:24.259987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:24.387996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:24.389514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:24.396387 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:24.409181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd25a23-3365-4ff1-a3a1-6759150cf793 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:24.595543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2539f4-9ad4-4018-af0d-06a1fb185988 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:24.604580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:24.605102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:24.605265 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:24.605380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:24.605659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] PATH_INFO: `/v3/projects/c4713904e2df4aeeb64b7abcbddbbc9f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:24.605772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:24.606052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:24.606340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] PATH_INFO: `/v3/projects/02f7cb109a3e42dba40d69f71d352664` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:24.607171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4713904e2df4aeeb64b7abcbddbbc9f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:24.608257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f7cb109a3e42dba40d69f71d352664)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:24.608890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:24.610213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:24.701602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4713904e2df4aeeb64b7abcbddbbc9f', 'request_id': 'req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.715769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4713904e2df4aeeb64b7abcbddbbc9f', 'request_id': 'req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.717330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:24.723869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4713904e2df4aeeb64b7abcbddbbc9f', 'request_id': 'req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.732503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f7cb109a3e42dba40d69f71d352664', 'request_id': 'req-bf89878b-f07c-469b-a8ba-ee85a3f96200'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.743249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] The token cache is being invalidate because project c4713904e2df4aeeb64b7abcbddbbc9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:24.743629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.744636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b32c628f-0b77-4664-a6cb-4327a9b2ba8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.745611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f7cb109a3e42dba40d69f71d352664', 'request_id': 'req-bf89878b-f07c-469b-a8ba-ee85a3f96200'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.747242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 979/1934] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:24 2022] DELETE /identity/v3/projects/c4713904e2df4aeeb64b7abcbddbbc9f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:24.752614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42be9bb-73f4-459a-b3f3-67e7b8b80626 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:24.753297 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42be9bb-73f4-459a-b3f3-67e7b8b80626 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:24.753987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42be9bb-73f4-459a-b3f3-67e7b8b80626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:24.755572 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f7cb109a3e42dba40d69f71d352664', 'request_id': 'req-bf89878b-f07c-469b-a8ba-ee85a3f96200'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.781693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e42be9bb-73f4-459a-b3f3-67e7b8b80626 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:24.783257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e42be9bb-73f4-459a-b3f3-67e7b8b80626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:24.791481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] The token cache is being invalidate because project 02f7cb109a3e42dba40d69f71d352664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:24.791780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.792469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf89878b-f07c-469b-a8ba-ee85a3f96200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:24.794122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 956/1935] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:24 2022] DELETE /identity/v3/projects/02f7cb109a3e42dba40d69f71d352664 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:24.801360 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97ea5-dd8f-421b-a12c-5ee4c74a1937 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:24.803112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97ea5-dd8f-421b-a12c-5ee4c74a1937 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:24.803400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97ea5-dd8f-421b-a12c-5ee4c74a1937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:24.843470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a1c97ea5-dd8f-421b-a12c-5ee4c74a1937 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:24.845710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a1c97ea5-dd8f-421b-a12c-5ee4c74a1937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:24.868657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 980/1936] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:24.876164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c30b-bc8d-45d5-8954-2004b7a156fc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:24.876586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c30b-bc8d-45d5-8954-2004b7a156fc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:24.876930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c30b-bc8d-45d5-8954-2004b7a156fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:24.907856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4c37c30b-bc8d-45d5-8954-2004b7a156fc None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:24.909423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4c37c30b-bc8d-45d5-8954-2004b7a156fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:24.920884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 981/1937] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:24.925694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37c30b-bc8d-45d5-8954-2004b7a156fc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:24.937703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37c30b-bc8d-45d5-8954-2004b7a156fc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:24.943803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:24.944423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:24.945256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:24.945537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:24.946347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:24.947191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:24.948913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:24.952752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 957/1938] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:24.954387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51524fea-fdd5-4ffe-9252-06253b0f5866 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:24.954677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51524fea-fdd5-4ffe-9252-06253b0f5866 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:24.954952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51524fea-fdd5-4ffe-9252-06253b0f5866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:24.975600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 982/1939] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:38:24.979808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:24.989642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-51524fea-fdd5-4ffe-9252-06253b0f5866 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:24.991673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-51524fea-fdd5-4ffe-9252-06253b0f5866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:24.996136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055b712-8140-4601-87ac-34653b80fffc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.001245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d289c-3741-4fee-9acd-dd5c71b03197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.001866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d289c-3741-4fee-9acd-dd5c71b03197 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.002147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d289c-3741-4fee-9acd-dd5c71b03197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.002420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d289c-3741-4fee-9acd-dd5c71b03197 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.002705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 958/1940] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:25.003114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d289c-3741-4fee-9acd-dd5c71b03197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.005113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d289c-3741-4fee-9acd-dd5c71b03197 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.010172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51524fea-fdd5-4ffe-9252-06253b0f5866 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.023740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51524fea-fdd5-4ffe-9252-06253b0f5866 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.024675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:25.031700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.032798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.033453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.034004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.035856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:25.037357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.039451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.053410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 983/1941] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:24 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:38:25.054362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d289c-3741-4fee-9acd-dd5c71b03197 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.059522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 959/1942] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:38:25.060527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.072392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52225506-75ea-4600-8a92-11e5544a5002 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.215289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d289c-3741-4fee-9acd-dd5c71b03197 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.221768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9236cde-e017-421b-bf91-86fcfe73dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.222815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9236cde-e017-421b-bf91-86fcfe73dc3c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.223104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9236cde-e017-421b-bf91-86fcfe73dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.223717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9236cde-e017-421b-bf91-86fcfe73dc3c admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.224390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9236cde-e017-421b-bf91-86fcfe73dc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.227187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9236cde-e017-421b-bf91-86fcfe73dc3c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.238265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b670c3-5117-48c7-b5f8-b41f590bdfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.239252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b670c3-5117-48c7-b5f8-b41f590bdfae admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.239660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b670c3-5117-48c7-b5f8-b41f590bdfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.240056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b670c3-5117-48c7-b5f8-b41f590bdfae admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.241293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b670c3-5117-48c7-b5f8-b41f590bdfae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.255842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b670c3-5117-48c7-b5f8-b41f590bdfae admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.262907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 984/1943] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:25 2022] POST /identity/v3/users => generated 290 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:25.277255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9236cde-e017-421b-bf91-86fcfe73dc3c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.298296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9236cde-e017-421b-bf91-86fcfe73dc3c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.299670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 960/1944] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:25 2022] POST /identity/v3/projects => generated 407 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:38:25.303794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b670c3-5117-48c7-b5f8-b41f590bdfae admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.381159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:25.453550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00dba-3b92-465b-99ad-3158d83b6fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.454197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00dba-3b92-465b-99ad-3158d83b6fae admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.454478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00dba-3b92-465b-99ad-3158d83b6fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.454750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00dba-3b92-465b-99ad-3158d83b6fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.455416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd00dba-3b92-465b-99ad-3158d83b6fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.457070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd00dba-3b92-465b-99ad-3158d83b6fae admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.457957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b670c3-5117-48c7-b5f8-b41f590bdfae admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.469334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938ecdf-f69f-4abd-88d1-9f833910a75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.471974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938ecdf-f69f-4abd-88d1-9f833910a75c admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.472267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938ecdf-f69f-4abd-88d1-9f833910a75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.473150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938ecdf-f69f-4abd-88d1-9f833910a75c admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.474145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938ecdf-f69f-4abd-88d1-9f833910a75c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.477620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 985/1945] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:25.481299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00dba-3b92-465b-99ad-3158d83b6fae admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.483980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938ecdf-f69f-4abd-88d1-9f833910a75c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.501031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00dba-3b92-465b-99ad-3158d83b6fae admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.518005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a0027-7d94-4a13-b6e5-0ce04e17cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.519765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a0027-7d94-4a13-b6e5-0ce04e17cb3b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.520060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a0027-7d94-4a13-b6e5-0ce04e17cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.520333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a0027-7d94-4a13-b6e5-0ce04e17cb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.522151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a0027-7d94-4a13-b6e5-0ce04e17cb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.526127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a0027-7d94-4a13-b6e5-0ce04e17cb3b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.549069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 986/1946] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:25.561020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a0027-7d94-4a13-b6e5-0ce04e17cb3b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.582500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a0027-7d94-4a13-b6e5-0ce04e17cb3b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.602153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7da6c-552d-418e-a42b-6b76af12ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.602833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7da6c-552d-418e-a42b-6b76af12ebe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.603106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7da6c-552d-418e-a42b-6b76af12ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.603373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7da6c-552d-418e-a42b-6b76af12ebe5 admin admin] PATH_INFO: `/v3/projects/07b9e359060f479aa51f47452cbae494/users/9992edc24f634892b7680a18518c8684/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.610717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 961/1947] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:25 2022] POST /identity/v3/users => generated 304 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:25.611721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938ecdf-f69f-4abd-88d1-9f833910a75c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.644860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7da6c-552d-418e-a42b-6b76af12ebe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b9e359060f479aa51f47452cbae494, user_id=9992edc24f634892b7680a18518c8684, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.646015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938ecdf-f69f-4abd-88d1-9f833910a75c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.649150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7da6c-552d-418e-a42b-6b76af12ebe5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.664097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a29ba-11d7-4102-87a1-81e9b074aa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.666885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a29ba-11d7-4102-87a1-81e9b074aa43 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.667001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a29ba-11d7-4102-87a1-81e9b074aa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.669854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a29ba-11d7-4102-87a1-81e9b074aa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.670717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 987/1948] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:25 2022] PUT /identity/v3/projects/07b9e359060f479aa51f47452cbae494/users/9992edc24f634892b7680a18518c8684/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:25.681157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a29ba-11d7-4102-87a1-81e9b074aa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.682065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cbd3-9c96-4c7a-a73b-ccbd53e968a9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.682352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cbd3-9c96-4c7a-a73b-ccbd53e968a9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.682619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cbd3-9c96-4c7a-a73b-ccbd53e968a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.682836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a29ba-11d7-4102-87a1-81e9b074aa43 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.715636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 962/1949] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:25.716142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a29ba-11d7-4102-87a1-81e9b074aa43 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.723915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1058cbd3-9c96-4c7a-a73b-ccbd53e968a9 None None] MFA Rules not processed for user `9992edc24f634892b7680a18518c8684`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:25.725370 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1058cbd3-9c96-4c7a-a73b-ccbd53e968a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:25.736104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a29ba-11d7-4102-87a1-81e9b074aa43 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.791080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:25.837027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 988/1950] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:25.841009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cbd3-9c96-4c7a-a73b-ccbd53e968a9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.845757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b21800-709f-4132-bd7a-aec81fb46283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.847708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21800-709f-4132-bd7a-aec81fb46283 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.848481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21800-709f-4132-bd7a-aec81fb46283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.849040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21800-709f-4132-bd7a-aec81fb46283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.851140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b21800-709f-4132-bd7a-aec81fb46283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.853976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b21800-709f-4132-bd7a-aec81fb46283 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.858165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cbd3-9c96-4c7a-a73b-ccbd53e968a9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.864389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ca674-9a04-4916-bd43-2ce58a4eef4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.865146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ca674-9a04-4916-bd43-2ce58a4eef4b admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.865458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ca674-9a04-4916-bd43-2ce58a4eef4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.865729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ca674-9a04-4916-bd43-2ce58a4eef4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.866892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ca674-9a04-4916-bd43-2ce58a4eef4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.868887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ca674-9a04-4916-bd43-2ce58a4eef4b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.871697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 963/1951] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:25.876136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b21800-709f-4132-bd7a-aec81fb46283 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.890438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b21800-709f-4132-bd7a-aec81fb46283 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:25.900010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e1bdc-9a90-4c3a-857d-4118775fe57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:25.900709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1bdc-9a90-4c3a-857d-4118775fe57b admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.900970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1bdc-9a90-4c3a-857d-4118775fe57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.901276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1bdc-9a90-4c3a-857d-4118775fe57b admin admin] PATH_INFO: `/v3/projects/231d3b133d3f47c49fe6ce10bf4881e0/users/42c36830f70b4d839cdd1c9246c68bc3/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:25.926210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e1bdc-9a90-4c3a-857d-4118775fe57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=231d3b133d3f47c49fe6ce10bf4881e0, user_id=42c36830f70b4d839cdd1c9246c68bc3, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:25.927479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e1bdc-9a90-4c3a-857d-4118775fe57b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:25.957734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 989/1952] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:25 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:25.960930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ca674-9a04-4916-bd43-2ce58a4eef4b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:25.985323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 964/1953] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:25 2022] PUT /identity/v3/projects/231d3b133d3f47c49fe6ce10bf4881e0/users/42c36830f70b4d839cdd1c9246c68bc3/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:25.997070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e905-f19c-4aac-82ef-af807c516a32 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:25.997070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e905-f19c-4aac-82ef-af807c516a32 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:25.997658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e905-f19c-4aac-82ef-af807c516a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.083233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ca674-9a04-4916-bd43-2ce58a4eef4b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:26.089907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e740e905-f19c-4aac-82ef-af807c516a32 None None] MFA Rules not processed for user `42c36830f70b4d839cdd1c9246c68bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:26.091345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e740e905-f19c-4aac-82ef-af807c516a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:26.240069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671fa01-ac0d-4e65-8d37-7c79625532f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:26.241941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671fa01-ac0d-4e65-8d37-7c79625532f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.242424 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671fa01-ac0d-4e65-8d37-7c79625532f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.242955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671fa01-ac0d-4e65-8d37-7c79625532f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.244290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671fa01-ac0d-4e65-8d37-7c79625532f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:26.246606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671fa01-ac0d-4e65-8d37-7c79625532f5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:26.309625 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 990/1954] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:25 2022] POST /identity/v3/users => generated 290 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:26.311611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc7bdb-5359-4d22-b51b-e8a4215fdbd7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.311918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc7bdb-5359-4d22-b51b-e8a4215fdbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.312237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc7bdb-5359-4d22-b51b-e8a4215fdbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.317077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 965/1955] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:25 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:26.318019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740e905-f19c-4aac-82ef-af807c516a32 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:26.330506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740e905-f19c-4aac-82ef-af807c516a32 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:26.336694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600e60-36d5-4a3a-b38b-429b5ca10acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:26.337517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600e60-36d5-4a3a-b38b-429b5ca10acc admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.337973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600e60-36d5-4a3a-b38b-429b5ca10acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.338366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600e60-36d5-4a3a-b38b-429b5ca10acc admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.339156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03600e60-36d5-4a3a-b38b-429b5ca10acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:26.341072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03600e60-36d5-4a3a-b38b-429b5ca10acc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:26.357716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:26.373894 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 966/1956] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:26.376514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600e60-36d5-4a3a-b38b-429b5ca10acc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:26.396650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600e60-36d5-4a3a-b38b-429b5ca10acc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:26.408567 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a092cc6-ee87-4e23-a210-f22540246951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:26.409699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8ddc7bdb-5359-4d22-b51b-e8a4215fdbd7 None None] MFA Rules not processed for user `411ff3f0306848ee91108343982806ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:26.411557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a092cc6-ee87-4e23-a210-f22540246951 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.411781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a092cc6-ee87-4e23-a210-f22540246951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.412135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a092cc6-ee87-4e23-a210-f22540246951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.414249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a092cc6-ee87-4e23-a210-f22540246951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:26.417752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8ddc7bdb-5359-4d22-b51b-e8a4215fdbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:26.419154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a092cc6-ee87-4e23-a210-f22540246951 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:26.493257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 967/1957] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:26 2022] POST /identity/v3/projects => generated 409 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:26.493257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a092cc6-ee87-4e23-a210-f22540246951 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:26.511515 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:26.621755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 991/1958] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:26.625859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc7bdb-5359-4d22-b51b-e8a4215fdbd7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:26.683386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc7bdb-5359-4d22-b51b-e8a4215fdbd7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:26.700075 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff95a8a-e339-49eb-b681-59d998479193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:26.703679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a092cc6-ee87-4e23-a210-f22540246951 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:26.703854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95a8a-e339-49eb-b681-59d998479193 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.704061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95a8a-e339-49eb-b681-59d998479193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.704285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95a8a-e339-49eb-b681-59d998479193 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.705090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff95a8a-e339-49eb-b681-59d998479193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:26.711306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff95a8a-e339-49eb-b681-59d998479193 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:26.714210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:26.714839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.715115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.715383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.719011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:26.722559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:26.750893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 968/1959] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:26.752023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:26.753519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:26.778902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c70ee-c03c-4b3e-8c19-08fc38ccac3f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:26.794494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c13b3-b95f-4fe5-98d7-6b3e0f7faeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:26.795204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c13b3-b95f-4fe5-98d7-6b3e0f7faeef admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.795431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c13b3-b95f-4fe5-98d7-6b3e0f7faeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.795652 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c13b3-b95f-4fe5-98d7-6b3e0f7faeef admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.796023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 992/1960] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:26 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:26.796890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c13b3-b95f-4fe5-98d7-6b3e0f7faeef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:26.798245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff95a8a-e339-49eb-b681-59d998479193 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:26.801424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c13b3-b95f-4fe5-98d7-6b3e0f7faeef admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:26.824983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff95a8a-e339-49eb-b681-59d998479193 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:26.840761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439d932-28df-4414-9815-f398de94dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:26.841624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439d932-28df-4414-9815-f398de94dc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.841907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439d932-28df-4414-9815-f398de94dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.842181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439d932-28df-4414-9815-f398de94dc7d admin admin] PATH_INFO: `/v3/projects/82af7a2dfe3b4a898e8eb76c9f81c3bc/users/5c7182024ece4d6f9c0720f3c80ae9d6/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.864479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439d932-28df-4414-9815-f398de94dc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82af7a2dfe3b4a898e8eb76c9f81c3bc, user_id=5c7182024ece4d6f9c0720f3c80ae9d6, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:26.868188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439d932-28df-4414-9815-f398de94dc7d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:26.875469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 969/1961] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:26 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:26.876716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c13b3-b95f-4fe5-98d7-6b3e0f7faeef admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:26.889849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 993/1962] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:26 2022] PUT /identity/v3/projects/82af7a2dfe3b4a898e8eb76c9f81c3bc/users/5c7182024ece4d6f9c0720f3c80ae9d6/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:26.891212 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439d932-28df-4414-9815-f398de94dc7d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:26.982986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439d932-28df-4414-9815-f398de94dc7d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:26.990059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e088a1-2ec6-4248-8fe1-ba547934e273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:26.990694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e088a1-2ec6-4248-8fe1-ba547934e273 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:26.990972 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e088a1-2ec6-4248-8fe1-ba547934e273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:26.991244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e088a1-2ec6-4248-8fe1-ba547934e273 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:26.993538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e088a1-2ec6-4248-8fe1-ba547934e273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:26.996975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e088a1-2ec6-4248-8fe1-ba547934e273 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.031499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 994/1963] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:26 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:27.032157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e866e6d-6b69-41ed-8fef-029f20690af0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.032447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e866e6d-6b69-41ed-8fef-029f20690af0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.033646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e866e6d-6b69-41ed-8fef-029f20690af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.058436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c13b3-b95f-4fe5-98d7-6b3e0f7faeef admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.079545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e4a1c-662a-4970-8ef3-ee5f4cd78895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.080179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e4a1c-662a-4970-8ef3-ee5f4cd78895 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.080456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e4a1c-662a-4970-8ef3-ee5f4cd78895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.080724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e4a1c-662a-4970-8ef3-ee5f4cd78895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.081401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e4a1c-662a-4970-8ef3-ee5f4cd78895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.082997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e4a1c-662a-4970-8ef3-ee5f4cd78895 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.105031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7e866e6d-6b69-41ed-8fef-029f20690af0 None None] MFA Rules not processed for user `5c7182024ece4d6f9c0720f3c80ae9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:27.106989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7e866e6d-6b69-41ed-8fef-029f20690af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:27.121258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 970/1964] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:27.125003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e4a1c-662a-4970-8ef3-ee5f4cd78895 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.136393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e4a1c-662a-4970-8ef3-ee5f4cd78895 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.142161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced11c8a-7414-4d62-add8-c9ad073742cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.142890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11c8a-7414-4d62-add8-c9ad073742cf admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.143173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11c8a-7414-4d62-add8-c9ad073742cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.143532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11c8a-7414-4d62-add8-c9ad073742cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.144356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced11c8a-7414-4d62-add8-c9ad073742cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.145829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced11c8a-7414-4d62-add8-c9ad073742cf admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.157091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 971/1965] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:27.157844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced11c8a-7414-4d62-add8-c9ad073742cf admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.186278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced11c8a-7414-4d62-add8-c9ad073742cf admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.192317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fe9f4-5b9a-4ebd-b09b-f150dca30cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.193087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe9f4-5b9a-4ebd-b09b-f150dca30cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.193279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe9f4-5b9a-4ebd-b09b-f150dca30cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.193555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe9f4-5b9a-4ebd-b09b-f150dca30cd7 admin admin] PATH_INFO: `/v3/projects/62367bbb4d474f098bb211918207f932/users/5c0810841ddb48cb83de29c218af4fbb/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.214138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fe9f4-5b9a-4ebd-b09b-f150dca30cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62367bbb4d474f098bb211918207f932, user_id=5c0810841ddb48cb83de29c218af4fbb, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.216806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fe9f4-5b9a-4ebd-b09b-f150dca30cd7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.256712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 972/1966] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:27 2022] PUT /identity/v3/projects/62367bbb4d474f098bb211918207f932/users/5c0810841ddb48cb83de29c218af4fbb/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:27.260102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fe9f4-5b9a-4ebd-b09b-f150dca30cd7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.269753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 995/1967] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:27.271631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e866e6d-6b69-41ed-8fef-029f20690af0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.290866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e866e6d-6b69-41ed-8fef-029f20690af0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.303728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:27.306570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:27.401891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad7360-8fda-45e8-ae7f-073331cd0901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.402630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad7360-8fda-45e8-ae7f-073331cd0901 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.403019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad7360-8fda-45e8-ae7f-073331cd0901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.404147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad7360-8fda-45e8-ae7f-073331cd0901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.404147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad7360-8fda-45e8-ae7f-073331cd0901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.405922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad7360-8fda-45e8-ae7f-073331cd0901 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.425750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 996/1968] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:27.425750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad7360-8fda-45e8-ae7f-073331cd0901 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.438186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fe9f4-5b9a-4ebd-b09b-f150dca30cd7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.440180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad7360-8fda-45e8-ae7f-073331cd0901 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.446986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab6986-18ba-4c0e-8aa8-2457a6940bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.447593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab6986-18ba-4c0e-8aa8-2457a6940bca admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.447965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab6986-18ba-4c0e-8aa8-2457a6940bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.448330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab6986-18ba-4c0e-8aa8-2457a6940bca admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.449049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab6986-18ba-4c0e-8aa8-2457a6940bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.463037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296439cc-99ab-4f73-ba05-5c2b6e40b500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.463266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab6986-18ba-4c0e-8aa8-2457a6940bca admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.463875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296439cc-99ab-4f73-ba05-5c2b6e40b500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.464155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296439cc-99ab-4f73-ba05-5c2b6e40b500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.464948 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296439cc-99ab-4f73-ba05-5c2b6e40b500 admin admin] PATH_INFO: `/v3/projects/42e30ac99cab4e50af392312a9333b5b/users/a7cfcc971ac04275adc58ab8d447a1c0/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.482338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296439cc-99ab-4f73-ba05-5c2b6e40b500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e30ac99cab4e50af392312a9333b5b, user_id=a7cfcc971ac04275adc58ab8d447a1c0, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.485348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296439cc-99ab-4f73-ba05-5c2b6e40b500 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.505203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 973/1969] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:27 2022] PUT /identity/v3/projects/42e30ac99cab4e50af392312a9333b5b/users/a7cfcc971ac04275adc58ab8d447a1c0/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:27.506089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296439cc-99ab-4f73-ba05-5c2b6e40b500 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.513047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 997/1970] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:27 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:27.514239 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab6986-18ba-4c0e-8aa8-2457a6940bca admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.591670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab6986-18ba-4c0e-8aa8-2457a6940bca admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.598072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f51092-306f-4ab4-b175-98a9a8ba99e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.598410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f51092-306f-4ab4-b175-98a9a8ba99e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.598771 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f51092-306f-4ab4-b175-98a9a8ba99e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.599373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f51092-306f-4ab4-b175-98a9a8ba99e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.600108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f51092-306f-4ab4-b175-98a9a8ba99e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.602503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f51092-306f-4ab4-b175-98a9a8ba99e6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.612040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296439cc-99ab-4f73-ba05-5c2b6e40b500 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.619397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd3d0-7fa2-4058-9be9-1568897cbe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.621489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd3d0-7fa2-4058-9be9-1568897cbe22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.621881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd3d0-7fa2-4058-9be9-1568897cbe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.622230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd3d0-7fa2-4058-9be9-1568897cbe22 admin admin] PATH_INFO: `/v3/projects/62367bbb4d474f098bb211918207f932/users/5c0810841ddb48cb83de29c218af4fbb/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.623537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 998/1971] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:27.624527 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f51092-306f-4ab4-b175-98a9a8ba99e6 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.625586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcd3d0-7fa2-4058-9be9-1568897cbe22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62367bbb4d474f098bb211918207f932, user_id=5c0810841ddb48cb83de29c218af4fbb, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.630345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcd3d0-7fa2-4058-9be9-1568897cbe22 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.636103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f51092-306f-4ab4-b175-98a9a8ba99e6 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.645103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae58bb-e937-4b5a-b59a-07721023602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.646338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae58bb-e937-4b5a-b59a-07721023602c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.648828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae58bb-e937-4b5a-b59a-07721023602c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.649167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae58bb-e937-4b5a-b59a-07721023602c admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.649875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae58bb-e937-4b5a-b59a-07721023602c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.651844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae58bb-e937-4b5a-b59a-07721023602c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.652772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 974/1972] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:27 2022] PUT /identity/v3/projects/62367bbb4d474f098bb211918207f932/users/5c0810841ddb48cb83de29c218af4fbb/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:27.662907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd3d0-7fa2-4058-9be9-1568897cbe22 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.685433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 999/1973] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:27 2022] POST /identity/v3/users => generated 296 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:27.689118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666eac58-d0a9-4ec4-a1e0-3a0e8f6f21b7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.689589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666eac58-d0a9-4ec4-a1e0-3a0e8f6f21b7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.690010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666eac58-d0a9-4ec4-a1e0-3a0e8f6f21b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.727785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-666eac58-d0a9-4ec4-a1e0-3a0e8f6f21b7 None None] MFA Rules not processed for user `5c0810841ddb48cb83de29c218af4fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:27.730206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-666eac58-d0a9-4ec4-a1e0-3a0e8f6f21b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:27.766851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd3d0-7fa2-4058-9be9-1568897cbe22 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.775609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacafefb-e818-4af0-8665-8656af92991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.777489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacafefb-e818-4af0-8665-8656af92991e admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.777785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacafefb-e818-4af0-8665-8656af92991e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.778058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacafefb-e818-4af0-8665-8656af92991e admin admin] PATH_INFO: `/v3/projects/42e30ac99cab4e50af392312a9333b5b/users/a7cfcc971ac04275adc58ab8d447a1c0/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.782647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacafefb-e818-4af0-8665-8656af92991e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e30ac99cab4e50af392312a9333b5b, user_id=a7cfcc971ac04275adc58ab8d447a1c0, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.786253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacafefb-e818-4af0-8665-8656af92991e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.804333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 975/1974] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:27 2022] PUT /identity/v3/projects/42e30ac99cab4e50af392312a9333b5b/users/a7cfcc971ac04275adc58ab8d447a1c0/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:27.805386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacafefb-e818-4af0-8665-8656af92991e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.814007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:27.828803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1000/1975] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:27 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:27.840811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666eac58-d0a9-4ec4-a1e0-3a0e8f6f21b7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:27.843022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-666eac58-d0a9-4ec4-a1e0-3a0e8f6f21b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:27.902228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:27.966207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacafefb-e818-4af0-8665-8656af92991e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.976235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabd66a-a093-48f2-9081-51d0e9e21d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:27.976867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabd66a-a093-48f2-9081-51d0e9e21d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:27.977141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabd66a-a093-48f2-9081-51d0e9e21d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:27.977461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabd66a-a093-48f2-9081-51d0e9e21d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:27.979361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabd66a-a093-48f2-9081-51d0e9e21d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:27.980408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabd66a-a093-48f2-9081-51d0e9e21d51 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:27.998051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 976/1976] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:27.999134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666eac58-d0a9-4ec4-a1e0-3a0e8f6f21b7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:27.999442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabd66a-a093-48f2-9081-51d0e9e21d51 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:28.015397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabd66a-a093-48f2-9081-51d0e9e21d51 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:28.022237 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fda5f-143c-4d50-a8e5-c8131ab3d2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:28.024041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fda5f-143c-4d50-a8e5-c8131ab3d2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:28.024041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fda5f-143c-4d50-a8e5-c8131ab3d2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:28.024475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fda5f-143c-4d50-a8e5-c8131ab3d2d0 admin admin] PATH_INFO: `/v3/projects/5cb59bcd349f4948a33af140b37549a0/users/232c6bf142b44d7b9fc063ab48cabcab/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:28.051195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2fda5f-143c-4d50-a8e5-c8131ab3d2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb59bcd349f4948a33af140b37549a0, user_id=232c6bf142b44d7b9fc063ab48cabcab, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:28.059222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2fda5f-143c-4d50-a8e5-c8131ab3d2d0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:28.068511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e05ba-dead-441a-9b55-c40909e59193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:28.070824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e05ba-dead-441a-9b55-c40909e59193 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:28.071276 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e05ba-dead-441a-9b55-c40909e59193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:28.071729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e05ba-dead-441a-9b55-c40909e59193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:28.100943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 977/1977] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:27 2022] PUT /identity/v3/projects/5cb59bcd349f4948a33af140b37549a0/users/232c6bf142b44d7b9fc063ab48cabcab/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:28.102081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fda5f-143c-4d50-a8e5-c8131ab3d2d0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:28.198054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e05ba-dead-441a-9b55-c40909e59193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:28.199898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e05ba-dead-441a-9b55-c40909e59193 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:28.221785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1001/1978] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:27 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:28.282809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fda5f-143c-4d50-a8e5-c8131ab3d2d0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:28.297853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eea61c-3b92-4d9b-9cf1-c963c40374ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:28.298707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea61c-3b92-4d9b-9cf1-c963c40374ae admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:28.299047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea61c-3b92-4d9b-9cf1-c963c40374ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:28.299445 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea61c-3b92-4d9b-9cf1-c963c40374ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:28.300103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eea61c-3b92-4d9b-9cf1-c963c40374ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:28.302888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eea61c-3b92-4d9b-9cf1-c963c40374ae admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:28.336420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 978/1979] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:28.340186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e05ba-dead-441a-9b55-c40909e59193 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:28.355677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e05ba-dead-441a-9b55-c40909e59193 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:28.375398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb5064-a2ca-4295-a5bf-7bdae1a0086d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:28.376726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb5064-a2ca-4295-a5bf-7bdae1a0086d admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:28.380027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb5064-a2ca-4295-a5bf-7bdae1a0086d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:28.381948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb5064-a2ca-4295-a5bf-7bdae1a0086d admin admin] PATH_INFO: `/v3/projects/5cb59bcd349f4948a33af140b37549a0/users/232c6bf142b44d7b9fc063ab48cabcab/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:28.393328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb5064-a2ca-4295-a5bf-7bdae1a0086d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb59bcd349f4948a33af140b37549a0, user_id=232c6bf142b44d7b9fc063ab48cabcab, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:28.400155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb5064-a2ca-4295-a5bf-7bdae1a0086d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:28.455907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133061d-d814-46cb-96b4-b7a3f81e7a3a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:28.456407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133061d-d814-46cb-96b4-b7a3f81e7a3a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:28.456619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133061d-d814-46cb-96b4-b7a3f81e7a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:28.457164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1002/1980] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:28 2022] PUT /identity/v3/projects/5cb59bcd349f4948a33af140b37549a0/users/232c6bf142b44d7b9fc063ab48cabcab/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:28.517494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6133061d-d814-46cb-96b4-b7a3f81e7a3a None None] MFA Rules not processed for user `232c6bf142b44d7b9fc063ab48cabcab`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:28.524321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6133061d-d814-46cb-96b4-b7a3f81e7a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:28.590151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb5064-a2ca-4295-a5bf-7bdae1a0086d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:28.673903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:28.681787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb5064-a2ca-4295-a5bf-7bdae1a0086d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:28.683227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 979/1981] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:28.691662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be270a00-1994-4920-85f7-39dbd1dac3b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:28.692367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be270a00-1994-4920-85f7-39dbd1dac3b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:28.692708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be270a00-1994-4920-85f7-39dbd1dac3b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:28.693047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be270a00-1994-4920-85f7-39dbd1dac3b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:28.697302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133061d-d814-46cb-96b4-b7a3f81e7a3a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:28.699345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6133061d-d814-46cb-96b4-b7a3f81e7a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:28.725578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133061d-d814-46cb-96b4-b7a3f81e7a3a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:28.738271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912dad2-3c79-4758-a2d9-131db262ffc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:28.738903 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912dad2-3c79-4758-a2d9-131db262ffc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:28.739206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912dad2-3c79-4758-a2d9-131db262ffc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:28.739476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912dad2-3c79-4758-a2d9-131db262ffc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:28.756557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912dad2-3c79-4758-a2d9-131db262ffc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:28.757983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912dad2-3c79-4758-a2d9-131db262ffc5 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:28.774399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be270a00-1994-4920-85f7-39dbd1dac3b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:28.775968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be270a00-1994-4920-85f7-39dbd1dac3b6 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:28.781645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 980/1982] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:28.786840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1003/1983] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:28 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:38:28.856559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912dad2-3c79-4758-a2d9-131db262ffc5 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:28.862579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:28.867065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912dad2-3c79-4758-a2d9-131db262ffc5 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:28.876169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dc5cf-fb95-4827-956a-18ccf7c1784a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:28.876776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc5cf-fb95-4827-956a-18ccf7c1784a service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:28.877052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc5cf-fb95-4827-956a-18ccf7c1784a service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:28.877381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc5cf-fb95-4827-956a-18ccf7c1784a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:28.973284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dc5cf-fb95-4827-956a-18ccf7c1784a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:28.974319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dc5cf-fb95-4827-956a-18ccf7c1784a service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:28.987923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 981/1984] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:35.035612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be270a00-1994-4920-85f7-39dbd1dac3b6 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:35.169258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be270a00-1994-4920-85f7-39dbd1dac3b6 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:35.183439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:35.186594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:35.187152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:35.187762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] PATH_INFO: `/v3/users/9992edc24f634892b7680a18518c8684` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:35.189503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9992edc24f634892b7680a18518c8684)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:35.193471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:35.321469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9992edc24f634892b7680a18518c8684'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:35.333153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9992edc24f634892b7680a18518c8684'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:35.336942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9992edc24f634892b7680a18518c8684'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:35.375268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9992edc24f634892b7680a18518c8684'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:35.376605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9992edc24f634892b7680a18518c8684'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:35.405346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1004/1985] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:35 2022] DELETE /identity/v3/users/9992edc24f634892b7680a18518c8684 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:35.408596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0bd9c-8e0f-4805-8405-3062230bf41a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:35.409143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0bd9c-8e0f-4805-8405-3062230bf41a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:35.409861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0bd9c-8e0f-4805-8405-3062230bf41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:35.455921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e6e0bd9c-8e0f-4805-8405-3062230bf41a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:35.457554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e6e0bd9c-8e0f-4805-8405-3062230bf41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:35.593717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 982/1986] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:35.624666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:35.624666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:35.666450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:35.750537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08537974-c1c1-4e6f-b4d8-82cacddb0857 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:35.755664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:35.756691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:35.757246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:35.757830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:35.780067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:35.781776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:35.796485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1005/1987] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:37.038738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948cca5-b460-4632-8e40-2001d09dcd14 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:37.039743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948cca5-b460-4632-8e40-2001d09dcd14 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:37.040242 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948cca5-b460-4632-8e40-2001d09dcd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:37.076399 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9948cca5-b460-4632-8e40-2001d09dcd14 None None] MFA Rules not processed for user `a7cfcc971ac04275adc58ab8d447a1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:37.078060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9948cca5-b460-4632-8e40-2001d09dcd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:37.178471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 983/1988] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:37.186093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:37.187728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:37.195568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948cca5-b460-4632-8e40-2001d09dcd14 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:37.199761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9948cca5-b460-4632-8e40-2001d09dcd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:37.216292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948cca5-b460-4632-8e40-2001d09dcd14 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:37.223781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:37.224846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:37.225130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:37.225592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] PATH_INFO: `/v3/projects/07b9e359060f479aa51f47452cbae494` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:37.226874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b9e359060f479aa51f47452cbae494)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:37.228926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:37.310311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:37.331845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566308e0-3994-482b-8c9c-19db4ca0cf53 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:37.344328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b9e359060f479aa51f47452cbae494', 'request_id': 'req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.352076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b9e359060f479aa51f47452cbae494', 'request_id': 'req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.359348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b9e359060f479aa51f47452cbae494', 'request_id': 'req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.375479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] The token cache is being invalidate because project 07b9e359060f479aa51f47452cbae494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:37.375840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.376485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.378266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 984/1989] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:37 2022] DELETE /identity/v3/projects/07b9e359060f479aa51f47452cbae494 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:37.382234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:37.384261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:37.424547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:37.426717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:37.427119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:37.427396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:37.437609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:37.463836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c56eb9-2d13-4c93-bb13-20a12d49b4c6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:37.470265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:37.471319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:37.471718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:37.473267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] PATH_INFO: `/v3/users/5c7182024ece4d6f9c0720f3c80ae9d6` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:37.473267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7182024ece4d6f9c0720f3c80ae9d6)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:37.475692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:37.548251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:37.549767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7182024ece4d6f9c0720f3c80ae9d6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.549914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:37.554310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:37.564650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7182024ece4d6f9c0720f3c80ae9d6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.565543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7182024ece4d6f9c0720f3c80ae9d6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.566518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1006/1990] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:38:37.586966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7182024ece4d6f9c0720f3c80ae9d6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.603743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-66414346-106d-4c7e-8f48-d6cf58c683fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7182024ece4d6f9c0720f3c80ae9d6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:37.617734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 985/1991] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:37 2022] DELETE /identity/v3/users/5c7182024ece4d6f9c0720f3c80ae9d6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:38.426646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:38.428190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:38.443235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3af8e-c455-42ca-9513-7134e59563d8 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:38.526874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:38.527672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:38.528372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:38.528865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] PATH_INFO: `/v3/users/42c36830f70b4d839cdd1c9246c68bc3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:38.530945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c36830f70b4d839cdd1c9246c68bc3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:38.537530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:38.603177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c36830f70b4d839cdd1c9246c68bc3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.612894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c36830f70b4d839cdd1c9246c68bc3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.613806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c36830f70b4d839cdd1c9246c68bc3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.627275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c36830f70b4d839cdd1c9246c68bc3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.628147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c36830f70b4d839cdd1c9246c68bc3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.646105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1007/1992] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:38 2022] DELETE /identity/v3/users/42c36830f70b4d839cdd1c9246c68bc3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:38.650358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d92f-3e71-4fe0-a0d7-4b15fc020e7e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:38.650837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d92f-3e71-4fe0-a0d7-4b15fc020e7e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:38.651156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d92f-3e71-4fe0-a0d7-4b15fc020e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:38.673219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6227d92f-3e71-4fe0-a0d7-4b15fc020e7e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:38.675473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6227d92f-3e71-4fe0-a0d7-4b15fc020e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:38.702971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:38.705073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:38.714230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:38.785016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 986/1993] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:38.804249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227d92f-3e71-4fe0-a0d7-4b15fc020e7e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:38.806362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6227d92f-3e71-4fe0-a0d7-4b15fc020e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:38.841612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04e526-9ef0-46bf-9b58-ffaa49e62e5a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:38.858468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:38.859402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:38.859890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:38.860280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] PATH_INFO: `/v3/projects/82af7a2dfe3b4a898e8eb76c9f81c3bc` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:38.862241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82af7a2dfe3b4a898e8eb76c9f81c3bc)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:38.867146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:38.892668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:38.911561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227d92f-3e71-4fe0-a0d7-4b15fc020e7e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:38.915020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5266f-41fb-4ac5-829d-f21d4b28b3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:38.915637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5266f-41fb-4ac5-829d-f21d4b28b3cd service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:38.915915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5266f-41fb-4ac5-829d-f21d4b28b3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:38.916185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5266f-41fb-4ac5-829d-f21d4b28b3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:38.926211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5266f-41fb-4ac5-829d-f21d4b28b3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:38.927654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5266f-41fb-4ac5-829d-f21d4b28b3cd service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:38.930407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82af7a2dfe3b4a898e8eb76c9f81c3bc', 'request_id': 'req-996677ff-7ff6-4480-9f52-1653ca024b28'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.938604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82af7a2dfe3b4a898e8eb76c9f81c3bc', 'request_id': 'req-996677ff-7ff6-4480-9f52-1653ca024b28'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.940138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 987/1994] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:38:38.945840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82af7a2dfe3b4a898e8eb76c9f81c3bc', 'request_id': 'req-996677ff-7ff6-4480-9f52-1653ca024b28'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.962751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] The token cache is being invalidate because project 82af7a2dfe3b4a898e8eb76c9f81c3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:38.963103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.963959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:38.966120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1008/1995] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:38 2022] DELETE /identity/v3/projects/82af7a2dfe3b4a898e8eb76c9f81c3bc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:38.969973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5266f-41fb-4ac5-829d-f21d4b28b3cd service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:38.981606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5266f-41fb-4ac5-829d-f21d4b28b3cd service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:39.085391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:39.086257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:39.086571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:39.086915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] PATH_INFO: `/v3/users/232c6bf142b44d7b9fc063ab48cabcab` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:39.088492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=232c6bf142b44d7b9fc063ab48cabcab)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:39.090637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:39.173048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '232c6bf142b44d7b9fc063ab48cabcab'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.184404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '232c6bf142b44d7b9fc063ab48cabcab'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.185373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '232c6bf142b44d7b9fc063ab48cabcab'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.202584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '232c6bf142b44d7b9fc063ab48cabcab'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.224285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:39.226845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '232c6bf142b44d7b9fc063ab48cabcab'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.234650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 988/1996] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:38 2022] DELETE /identity/v3/users/232c6bf142b44d7b9fc063ab48cabcab => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:39.309567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:39.311430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:39.331660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996677ff-7ff6-4480-9f52-1653ca024b28 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:39.441276 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:39.442618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:39.443001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:39.443465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] PATH_INFO: `/v3/projects/231d3b133d3f47c49fe6ce10bf4881e0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:39.445500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231d3b133d3f47c49fe6ce10bf4881e0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:39.448365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:39.571445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231d3b133d3f47c49fe6ce10bf4881e0', 'request_id': 'req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.587613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231d3b133d3f47c49fe6ce10bf4881e0', 'request_id': 'req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.607362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231d3b133d3f47c49fe6ce10bf4881e0', 'request_id': 'req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.630794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] The token cache is being invalidate because project 231d3b133d3f47c49fe6ce10bf4881e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:39.631338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.632232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.637971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1009/1997] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:39 2022] DELETE /identity/v3/projects/231d3b133d3f47c49fe6ce10bf4881e0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:39.638722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:39.640953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:39.659706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfadac-32cd-44c3-92a8-d79de540d544 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:39.844491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:39.845247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:39.845544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:39.845826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] PATH_INFO: `/v3/users/5c0810841ddb48cb83de29c218af4fbb` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:39.847470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c0810841ddb48cb83de29c218af4fbb)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:39.850722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:39.933055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c0810841ddb48cb83de29c218af4fbb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.949522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c0810841ddb48cb83de29c218af4fbb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.949762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c0810841ddb48cb83de29c218af4fbb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:39.982258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:39.983716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:40.003368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c0810841ddb48cb83de29c218af4fbb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.019669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab6d0-3b21-44d4-aa5d-25549fb2efb8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:40.039572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:40.041961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.043713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.049198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] PATH_INFO: `/v3/projects/5cb59bcd349f4948a33af140b37549a0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.052129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb59bcd349f4948a33af140b37549a0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:40.053715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c75dfa11-d976-4357-b2da-a6d49f55b769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c0810841ddb48cb83de29c218af4fbb'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.055824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:40.079899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 989/1998] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:39 2022] DELETE /identity/v3/users/5c0810841ddb48cb83de29c218af4fbb => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:40.144965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:40.171502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb59bcd349f4948a33af140b37549a0', 'request_id': 'req-a802dc5f-3eab-4519-8625-67e26ecb9ed8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.184417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb59bcd349f4948a33af140b37549a0', 'request_id': 'req-a802dc5f-3eab-4519-8625-67e26ecb9ed8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.200380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb59bcd349f4948a33af140b37549a0', 'request_id': 'req-a802dc5f-3eab-4519-8625-67e26ecb9ed8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.230781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] The token cache is being invalidate because project 5cb59bcd349f4948a33af140b37549a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:40.231431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.232379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a802dc5f-3eab-4519-8625-67e26ecb9ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.241800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1010/1999] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:39 2022] DELETE /identity/v3/projects/5cb59bcd349f4948a33af140b37549a0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:40.242917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c72b57-3afe-42c8-b2af-ab8606856fdd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.243063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c72b57-3afe-42c8-b2af-ab8606856fdd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.243310 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c72b57-3afe-42c8-b2af-ab8606856fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.298403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-27c72b57-3afe-42c8-b2af-ab8606856fdd None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:40.300053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-27c72b57-3afe-42c8-b2af-ab8606856fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:40.439470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d5b4-d41e-4270-9923-7b4a78b9c565 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.439762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d5b4-d41e-4270-9923-7b4a78b9c565 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.440038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d5b4-d41e-4270-9923-7b4a78b9c565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.443730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 990/2000] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:40.462393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-eb83d5b4-d41e-4270-9923-7b4a78b9c565 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:40.463833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-eb83d5b4-d41e-4270-9923-7b4a78b9c565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:40.473783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1011/2001] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:40.479533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c72b57-3afe-42c8-b2af-ab8606856fdd None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:40.490617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c72b57-3afe-42c8-b2af-ab8606856fdd None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:40.496526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:40.497163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.497474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.497742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.498433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:40.498737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:40.500334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:40.510089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:40.520330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83d5b4-d41e-4270-9923-7b4a78b9c565 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:40.533542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 991/2002] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:38:40.537505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83d5b4-d41e-4270-9923-7b4a78b9c565 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:40.553316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce2bc9-9074-4df2-b9f5-1d55e7a56712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:40.554087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2bc9-9074-4df2-b9f5-1d55e7a56712 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.554460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2bc9-9074-4df2-b9f5-1d55e7a56712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.554826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2bc9-9074-4df2-b9f5-1d55e7a56712 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.555765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce2bc9-9074-4df2-b9f5-1d55e7a56712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:40.557916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce2bc9-9074-4df2-b9f5-1d55e7a56712 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:40.601222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1012/2003] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:40 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:40.605434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:40.618673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f709a32-d552-4ba1-9e0e-07d84e032a8e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:40.697130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e6924-c892-40b2-8886-6794ea59fbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:40.697864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e6924-c892-40b2-8886-6794ea59fbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.698142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e6924-c892-40b2-8886-6794ea59fbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.698411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e6924-c892-40b2-8886-6794ea59fbb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.700345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e6924-c892-40b2-8886-6794ea59fbb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:40.702419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e6924-c892-40b2-8886-6794ea59fbb4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:40.730183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 992/2004] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:40 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:40.731738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce2bc9-9074-4df2-b9f5-1d55e7a56712 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:40.746580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce2bc9-9074-4df2-b9f5-1d55e7a56712 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:40.766858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec01a75-99ff-449e-ac65-665ea88bbd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:40.767490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec01a75-99ff-449e-ac65-665ea88bbd4d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.769692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec01a75-99ff-449e-ac65-665ea88bbd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.770229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec01a75-99ff-449e-ac65-665ea88bbd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.771150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec01a75-99ff-449e-ac65-665ea88bbd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:40.773788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec01a75-99ff-449e-ac65-665ea88bbd4d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:40.790978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1013/2005] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:40.794797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e6924-c892-40b2-8886-6794ea59fbb4 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:40.806959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e6924-c892-40b2-8886-6794ea59fbb4 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:40.812207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:40.813127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.813617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.814526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.815253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:40.817176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:40.832977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 993/2006] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:38:40.834836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec01a75-99ff-449e-ac65-665ea88bbd4d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:40.845501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec01a75-99ff-449e-ac65-665ea88bbd4d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:40.857836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ba0d8-8d4c-4935-840f-0f1337813552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:40.858835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba0d8-8d4c-4935-840f-0f1337813552 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.859269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba0d8-8d4c-4935-840f-0f1337813552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.859647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba0d8-8d4c-4935-840f-0f1337813552 admin admin] PATH_INFO: `/v3/projects/b2e8252d092e4087bf47db45b137afc2/users/77bb717dc2404960850221d75817a221/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.869742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:40.872271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:40.891438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166e60c-71a6-4a23-ab3e-32c5fc73ba76 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:40.906880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ba0d8-8d4c-4935-840f-0f1337813552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e8252d092e4087bf47db45b137afc2, user_id=77bb717dc2404960850221d75817a221, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:40.910043 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ba0d8-8d4c-4935-840f-0f1337813552 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:40.914555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:40.915950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.916374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.916834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] PATH_INFO: `/v3/projects/62367bbb4d474f098bb211918207f932` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.924096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62367bbb4d474f098bb211918207f932)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:40.926080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:40.935353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1014/2007] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:40 2022] PUT /identity/v3/projects/b2e8252d092e4087bf47db45b137afc2/users/77bb717dc2404960850221d75817a221/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:40.940183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5f90-bb9e-43c3-ab83-41a795f871f0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:40.940466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5f90-bb9e-43c3-ab83-41a795f871f0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:40.940737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5f90-bb9e-43c3-ab83-41a795f871f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:40.958982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e95a5f90-bb9e-43c3-ab83-41a795f871f0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:40.960722 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e95a5f90-bb9e-43c3-ab83-41a795f871f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:40.984665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62367bbb4d474f098bb211918207f932', 'request_id': 'req-bcb18718-7095-4ea3-9858-fefd4baec673'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.990810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62367bbb4d474f098bb211918207f932', 'request_id': 'req-bcb18718-7095-4ea3-9858-fefd4baec673'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:40.998062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62367bbb4d474f098bb211918207f932', 'request_id': 'req-bcb18718-7095-4ea3-9858-fefd4baec673'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:41.014591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] The token cache is being invalidate because project 62367bbb4d474f098bb211918207f932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:41.014988 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:41.015656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bcb18718-7095-4ea3-9858-fefd4baec673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:41.017744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 994/2008] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:40 2022] DELETE /identity/v3/projects/62367bbb4d474f098bb211918207f932 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:41.022088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0114917-e81e-4bbb-9efc-e9395b644671 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.025491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0114917-e81e-4bbb-9efc-e9395b644671 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.026751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0114917-e81e-4bbb-9efc-e9395b644671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.052353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1015/2009] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:41.054386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d0114917-e81e-4bbb-9efc-e9395b644671 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:41.055068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d0114917-e81e-4bbb-9efc-e9395b644671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:41.061462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a5f90-bb9e-43c3-ab83-41a795f871f0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:41.062952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e95a5f90-bb9e-43c3-ab83-41a795f871f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:41.119742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 995/2010] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:41.127183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a5f90-bb9e-43c3-ab83-41a795f871f0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:41.128139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22d35a-ba9c-46ef-8590-0b3d331589cf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.128427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22d35a-ba9c-46ef-8590-0b3d331589cf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.128703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22d35a-ba9c-46ef-8590-0b3d331589cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.137594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:41.138291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:41.156536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0b22d35a-ba9c-46ef-8590-0b3d331589cf None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:41.157799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0b22d35a-ba9c-46ef-8590-0b3d331589cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:41.174014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 996/2011] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:38:41.181767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22d35a-ba9c-46ef-8590-0b3d331589cf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:41.193681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22d35a-ba9c-46ef-8590-0b3d331589cf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:41.194635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916272a-c745-4cf6-80cf-bb9230940988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:41.195902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916272a-c745-4cf6-80cf-bb9230940988 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.196326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916272a-c745-4cf6-80cf-bb9230940988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.196734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916272a-c745-4cf6-80cf-bb9230940988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.200231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:41.201106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.201423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.201697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.202348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:41.202905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:41.204675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:41.210784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916272a-c745-4cf6-80cf-bb9230940988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:41.212664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916272a-c745-4cf6-80cf-bb9230940988 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:41.223889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1016/2012] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:38:41.224706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916272a-c745-4cf6-80cf-bb9230940988 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:41.229401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 997/2013] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:38:41.235198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916272a-c745-4cf6-80cf-bb9230940988 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:41.239886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f2e0e-bccd-411e-a24e-1bc08d8d19af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:41.240591 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f2e0e-bccd-411e-a24e-1bc08d8d19af admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.240903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f2e0e-bccd-411e-a24e-1bc08d8d19af admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.241223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f2e0e-bccd-411e-a24e-1bc08d8d19af admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.241924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f2e0e-bccd-411e-a24e-1bc08d8d19af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:41.243907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f2e0e-bccd-411e-a24e-1bc08d8d19af admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:41.287003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1017/2014] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:41 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:41.291890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:41.303010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0766a4-acb3-43de-8e10-3cf229382c2b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:41.379795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeea770-5d1b-4e8f-bb3b-cb0636334851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:41.380819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeea770-5d1b-4e8f-bb3b-cb0636334851 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.380819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeea770-5d1b-4e8f-bb3b-cb0636334851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.381117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeea770-5d1b-4e8f-bb3b-cb0636334851 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.382457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeea770-5d1b-4e8f-bb3b-cb0636334851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:41.384388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeea770-5d1b-4e8f-bb3b-cb0636334851 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:41.411196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 998/2015] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:41 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:41.412486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f2e0e-bccd-411e-a24e-1bc08d8d19af admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:41.424667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f2e0e-bccd-411e-a24e-1bc08d8d19af admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:41.429364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e21657-12d7-4cda-b3d2-245885735112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:41.429961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e21657-12d7-4cda-b3d2-245885735112 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.430233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e21657-12d7-4cda-b3d2-245885735112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.430500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e21657-12d7-4cda-b3d2-245885735112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.431233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e21657-12d7-4cda-b3d2-245885735112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:41.432931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e21657-12d7-4cda-b3d2-245885735112 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:41.445262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1018/2016] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:38:41.449251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeea770-5d1b-4e8f-bb3b-cb0636334851 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:41.459508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeea770-5d1b-4e8f-bb3b-cb0636334851 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:41.465111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cc37e-0628-43fd-8020-f1e6ccaee64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:41.465747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc37e-0628-43fd-8020-f1e6ccaee64b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.466023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc37e-0628-43fd-8020-f1e6ccaee64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.466289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc37e-0628-43fd-8020-f1e6ccaee64b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.466928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cc37e-0628-43fd-8020-f1e6ccaee64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:41.468507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cc37e-0628-43fd-8020-f1e6ccaee64b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:41.481841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 999/2017] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:41.486338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e21657-12d7-4cda-b3d2-245885735112 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:41.497989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e21657-12d7-4cda-b3d2-245885735112 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:41.505095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:41.506065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.507065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.507417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] PATH_INFO: `/v3/projects/565ccb02b2d74ca9a5c443aebe4b69db/users/8f49a52c17954d7fa609c65c560fd36c/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.525881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=565ccb02b2d74ca9a5c443aebe4b69db, user_id=8f49a52c17954d7fa609c65c560fd36c, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:41.529672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:41.573535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1019/2018] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:41 2022] PUT /identity/v3/projects/565ccb02b2d74ca9a5c443aebe4b69db/users/8f49a52c17954d7fa609c65c560fd36c/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:41.578609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910b29c-261d-457f-ab5f-79eb33adc577 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.581556 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910b29c-261d-457f-ab5f-79eb33adc577 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.582103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910b29c-261d-457f-ab5f-79eb33adc577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.655678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4910b29c-261d-457f-ab5f-79eb33adc577 None None] MFA Rules not processed for user `8f49a52c17954d7fa609c65c560fd36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:41.656931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4910b29c-261d-457f-ab5f-79eb33adc577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:41.773387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1000/2019] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:41.786637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:41.792505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:41.947872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da627-547c-4875-aa7f-391b628b72bf admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:41.952586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce851cf0509947e783dbd5b4809bc897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:41.953781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:41.953781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:41.953972 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:41.967473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:41.969103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:41.986263 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1020/2020] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:42.065306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eefda-0f09-4eb8-8db7-8db40442406e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:42.065998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eefda-0f09-4eb8-8db7-8db40442406e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:42.066289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eefda-0f09-4eb8-8db7-8db40442406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:42.145651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bb0eefda-0f09-4eb8-8db7-8db40442406e None None] MFA Rules not processed for user `77bb717dc2404960850221d75817a221`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:42.146960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bb0eefda-0f09-4eb8-8db7-8db40442406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:42.220567 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:42.263487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1001/2021] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:42.267981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:42.269467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:42.340886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:42.366560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef58e-caab-4922-802b-24d825df1db2 service glance] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:42.380024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58937f-aa6f-481e-9c2e-048f97828789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:42.391173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58937f-aa6f-481e-9c2e-048f97828789 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:42.391173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58937f-aa6f-481e-9c2e-048f97828789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:42.391173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58937f-aa6f-481e-9c2e-048f97828789 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:42.391173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58937f-aa6f-481e-9c2e-048f97828789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:42.391173 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58937f-aa6f-481e-9c2e-048f97828789 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:42.436683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1021/2022] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:42 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:42.437876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0eefda-0f09-4eb8-8db7-8db40442406e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:42.487732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0eefda-0f09-4eb8-8db7-8db40442406e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:42.645495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d90a8-e568-4fff-a792-5b948d539c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:42.645495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d90a8-e568-4fff-a792-5b948d539c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:42.645495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d90a8-e568-4fff-a792-5b948d539c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:42.645495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d90a8-e568-4fff-a792-5b948d539c83 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:42.646356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d90a8-e568-4fff-a792-5b948d539c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:42.674803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d90a8-e568-4fff-a792-5b948d539c83 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:42.730805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1002/2023] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:42 2022] POST /identity/v3/users => generated 293 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:38:42.734182 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58937f-aa6f-481e-9c2e-048f97828789 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:42.760650 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58937f-aa6f-481e-9c2e-048f97828789 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:42.772065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b9d7d-a9ba-4823-9832-876208ac2e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:42.773254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b9d7d-a9ba-4823-9832-876208ac2e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:42.773539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b9d7d-a9ba-4823-9832-876208ac2e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:42.773807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b9d7d-a9ba-4823-9832-876208ac2e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:42.774453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b9d7d-a9ba-4823-9832-876208ac2e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:42.776057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b9d7d-a9ba-4823-9832-876208ac2e3d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:42.794001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1022/2024] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:38:42.798422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d90a8-e568-4fff-a792-5b948d539c83 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:42.813501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d90a8-e568-4fff-a792-5b948d539c83 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:42.819865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a4158-e474-460f-a2b0-6e4f7031cbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:42.820681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a4158-e474-460f-a2b0-6e4f7031cbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:42.821025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a4158-e474-460f-a2b0-6e4f7031cbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:42.821415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a4158-e474-460f-a2b0-6e4f7031cbd1 admin admin] PATH_INFO: `/v3/projects/363961cc0c42473c8d92ae17ed4fa667/users/9c61a19732ea4c32a120b4f4d9f35f7e/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:42.846561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a4158-e474-460f-a2b0-6e4f7031cbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363961cc0c42473c8d92ae17ed4fa667, user_id=9c61a19732ea4c32a120b4f4d9f35f7e, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:42.849432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a4158-e474-460f-a2b0-6e4f7031cbd1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:42.865795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1003/2025] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:42 2022] PUT /identity/v3/projects/363961cc0c42473c8d92ae17ed4fa667/users/9c61a19732ea4c32a120b4f4d9f35f7e/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:42.870416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b9d7d-a9ba-4823-9832-876208ac2e3d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:42.880551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b9d7d-a9ba-4823-9832-876208ac2e3d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:42.939174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f2d79-d039-4fa7-a3a2-0c8f1e1ed03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:42.940167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2d79-d039-4fa7-a3a2-0c8f1e1ed03f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:42.940510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2d79-d039-4fa7-a3a2-0c8f1e1ed03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:42.940841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2d79-d039-4fa7-a3a2-0c8f1e1ed03f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:42.941649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f2d79-d039-4fa7-a3a2-0c8f1e1ed03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:42.943518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f2d79-d039-4fa7-a3a2-0c8f1e1ed03f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:42.962172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a4158-e474-460f-a2b0-6e4f7031cbd1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:42.968986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1023/2026] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:38:42.974889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a4158-e474-460f-a2b0-6e4f7031cbd1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:42.980773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:42.981501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:42.981781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:42.982123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] PATH_INFO: `/v3/projects/363961cc0c42473c8d92ae17ed4fa667/users/9c61a19732ea4c32a120b4f4d9f35f7e/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:42.984200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363961cc0c42473c8d92ae17ed4fa667, user_id=9c61a19732ea4c32a120b4f4d9f35f7e, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:42.986987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:43.002935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1004/2027] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:42 2022] PUT /identity/v3/projects/363961cc0c42473c8d92ae17ed4fa667/users/9c61a19732ea4c32a120b4f4d9f35f7e/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:43.932413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57081e95-2ea0-42aa-91fd-32ef3e3865f2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:43.933334 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57081e95-2ea0-42aa-91fd-32ef3e3865f2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:43.933761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57081e95-2ea0-42aa-91fd-32ef3e3865f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:43.965973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-57081e95-2ea0-42aa-91fd-32ef3e3865f2 None None] MFA Rules not processed for user `9c61a19732ea4c32a120b4f4d9f35f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:43.967187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-57081e95-2ea0-42aa-91fd-32ef3e3865f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:44.065014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1024/2028] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:44.072453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:44.075203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:44.176043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6343-a076-49d8-b163-558056bd0cc0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:44.188929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:44.189767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:44.190044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:44.190316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:44.266451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:44.283017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:44.284464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:44.295333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1005/2029] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:45.691260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57081e95-2ea0-42aa-91fd-32ef3e3865f2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:45.692661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-57081e95-2ea0-42aa-91fd-32ef3e3865f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:45.704467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57081e95-2ea0-42aa-91fd-32ef3e3865f2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:45.707940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b9dbe-27f0-47b4-8b07-2222dbd32a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:45.708525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b9dbe-27f0-47b4-8b07-2222dbd32a82 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:45.708805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b9dbe-27f0-47b4-8b07-2222dbd32a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:45.709074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b9dbe-27f0-47b4-8b07-2222dbd32a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:45.718859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b9dbe-27f0-47b4-8b07-2222dbd32a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:45.720303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b9dbe-27f0-47b4-8b07-2222dbd32a82 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:45.731853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1025/2030] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:38:53.369639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:53.371259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:53.436067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5386a-e499-4fb8-b5d5-8ccd1eafad86 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:53.440543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:53.441201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:53.441507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:53.441775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] PATH_INFO: `/v3/users/258ccb4bb15a4747b4c17d2a26645888` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:53.442884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258ccb4bb15a4747b4c17d2a26645888)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:53.445363 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:53.492915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258ccb4bb15a4747b4c17d2a26645888'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.500671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258ccb4bb15a4747b4c17d2a26645888'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.501517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258ccb4bb15a4747b4c17d2a26645888'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.509443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258ccb4bb15a4747b4c17d2a26645888'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.521372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f7d8194-fe38-49e9-aa5d-30b3b9b934c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258ccb4bb15a4747b4c17d2a26645888'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.527370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1006/2031] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:38:53 2022] DELETE /identity/v3/users/258ccb4bb15a4747b4c17d2a26645888 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:53.703855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b9dbe-27f0-47b4-8b07-2222dbd32a82 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:53.720892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b9dbe-27f0-47b4-8b07-2222dbd32a82 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:53.743934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:53.791086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:53.791719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:53.791981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:53.792245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] PATH_INFO: `/v3/projects/18c99d576d184b04b933db13eabb8c5e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:53.793383 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18c99d576d184b04b933db13eabb8c5e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:53.795281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:53.853375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18c99d576d184b04b933db13eabb8c5e', 'request_id': 'req-d9093fc4-51b3-4e0f-9959-3552cca29ab6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.858952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18c99d576d184b04b933db13eabb8c5e', 'request_id': 'req-d9093fc4-51b3-4e0f-9959-3552cca29ab6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.866710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18c99d576d184b04b933db13eabb8c5e', 'request_id': 'req-d9093fc4-51b3-4e0f-9959-3552cca29ab6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.880005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] The token cache is being invalidate because project 18c99d576d184b04b933db13eabb8c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:38:53.880244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.880828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d9093fc4-51b3-4e0f-9959-3552cca29ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:38:53.882980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1026/2032] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:38:53 2022] DELETE /identity/v3/projects/18c99d576d184b04b933db13eabb8c5e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:38:53.887845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb366831-c81b-410a-b1da-6b29f46b1f0a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:53.888127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb366831-c81b-410a-b1da-6b29f46b1f0a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:53.888400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb366831-c81b-410a-b1da-6b29f46b1f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:53.911681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fb366831-c81b-410a-b1da-6b29f46b1f0a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:53.912912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fb366831-c81b-410a-b1da-6b29f46b1f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:54.000521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1007/2033] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:54.009187 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21948-c9d1-4d4d-a032-49751bfb1586 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.009504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21948-c9d1-4d4d-a032-49751bfb1586 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.009779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21948-c9d1-4d4d-a032-49751bfb1586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.029369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-45e21948-c9d1-4d4d-a032-49751bfb1586 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:54.030570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-45e21948-c9d1-4d4d-a032-49751bfb1586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:54.039015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1027/2034] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:54.043314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb366831-c81b-410a-b1da-6b29f46b1f0a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:54.053510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb366831-c81b-410a-b1da-6b29f46b1f0a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:54.058388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:54.059168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.059446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.059714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.060363 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:38:54.060659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:54.062355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:54.064004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:54.078319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1008/2035] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:38:54.081014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e21948-c9d1-4d4d-a032-49751bfb1586 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:54.091158 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e21948-c9d1-4d4d-a032-49751bfb1586 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:54.096180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09812c4c-3719-4928-ac47-d2d45111849b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:54.096778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812c4c-3719-4928-ac47-d2d45111849b admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.097053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812c4c-3719-4928-ac47-d2d45111849b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.097414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812c4c-3719-4928-ac47-d2d45111849b admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.098043 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09812c4c-3719-4928-ac47-d2d45111849b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:54.100049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09812c4c-3719-4928-ac47-d2d45111849b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:54.137074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1028/2036] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:38:54 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:54.138664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:54.150701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131def6-e8f7-42fb-8d47-e4733d877164 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:54.222236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f95ab-8ee6-4e15-97aa-3df52d6d235f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:54.222875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f95ab-8ee6-4e15-97aa-3df52d6d235f admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.223152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f95ab-8ee6-4e15-97aa-3df52d6d235f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.223436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f95ab-8ee6-4e15-97aa-3df52d6d235f admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.224055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f95ab-8ee6-4e15-97aa-3df52d6d235f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:54.226679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f95ab-8ee6-4e15-97aa-3df52d6d235f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:54.275521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1009/2037] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:38:54 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:38:54.276784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09812c4c-3719-4928-ac47-d2d45111849b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:54.290271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09812c4c-3719-4928-ac47-d2d45111849b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:54.295112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1db74-6283-41e1-856b-a277a689b83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:54.295802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1db74-6283-41e1-856b-a277a689b83f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.296146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1db74-6283-41e1-856b-a277a689b83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.296483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1db74-6283-41e1-856b-a277a689b83f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.297210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1db74-6283-41e1-856b-a277a689b83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:54.298974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1db74-6283-41e1-856b-a277a689b83f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:54.311313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1029/2038] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:54.315772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f95ab-8ee6-4e15-97aa-3df52d6d235f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:54.326440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f95ab-8ee6-4e15-97aa-3df52d6d235f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:54.332313 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318960b-b35a-438c-8fa5-97ec39471e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:54.332932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318960b-b35a-438c-8fa5-97ec39471e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.333207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318960b-b35a-438c-8fa5-97ec39471e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.333503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318960b-b35a-438c-8fa5-97ec39471e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.334151 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318960b-b35a-438c-8fa5-97ec39471e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:54.335761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318960b-b35a-438c-8fa5-97ec39471e70 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:54.346378 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1010/2039] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:38:54.351242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1db74-6283-41e1-856b-a277a689b83f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:54.364041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1db74-6283-41e1-856b-a277a689b83f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:54.370894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:54.371712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.371991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.372280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] PATH_INFO: `/v3/projects/9aa8759e166c497d9fdba8e566fd98ec/users/f36fd12e187d495bae90c61d17227ac3/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.398459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8759e166c497d9fdba8e566fd98ec, user_id=f36fd12e187d495bae90c61d17227ac3, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:54.401257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:54.415769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1030/2040] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:38:54 2022] PUT /identity/v3/projects/9aa8759e166c497d9fdba8e566fd98ec/users/f36fd12e187d495bae90c61d17227ac3/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:38:54.426004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa159e-da78-4b5a-816b-4b052d13272b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.426514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa159e-da78-4b5a-816b-4b052d13272b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.426624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa159e-da78-4b5a-816b-4b052d13272b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.454302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f7aa159e-da78-4b5a-816b-4b052d13272b None None] MFA Rules not processed for user `f36fd12e187d495bae90c61d17227ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:38:54.455528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f7aa159e-da78-4b5a-816b-4b052d13272b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:54.522727 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1011/2041] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:38:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:38:54.534897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:38:54.536590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:38:54.589006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:38:54.612851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d2d7a-b655-4051-a0b3-9ee8b8de81c5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:38:54.618814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fa8ef-18ce-443d-b8e3-1fd238e4cc59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:38:54.619414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa8ef-18ce-443d-b8e3-1fd238e4cc59 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:38:54.619688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa8ef-18ce-443d-b8e3-1fd238e4cc59 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:38:54.619954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa8ef-18ce-443d-b8e3-1fd238e4cc59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:38:54.627444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fa8ef-18ce-443d-b8e3-1fd238e4cc59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:38:54.629110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fa8ef-18ce-443d-b8e3-1fd238e4cc59 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:38:54.638676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1031/2042] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:38:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:02.539060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa159e-da78-4b5a-816b-4b052d13272b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:02.540550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f7aa159e-da78-4b5a-816b-4b052d13272b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:02.611929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa159e-da78-4b5a-816b-4b052d13272b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:02.617702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:02.618589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:02.618938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:02.619309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] PATH_INFO: `/v3/users/77bb717dc2404960850221d75817a221` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:02.620523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=77bb717dc2404960850221d75817a221)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:02.622684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:02.625184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:02.674814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77bb717dc2404960850221d75817a221'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:02.683004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77bb717dc2404960850221d75817a221'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:02.683563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77bb717dc2404960850221d75817a221'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:02.690733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77bb717dc2404960850221d75817a221'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:02.703042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77bb717dc2404960850221d75817a221'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:02.708981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1012/2043] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:02 2022] DELETE /identity/v3/users/77bb717dc2404960850221d75817a221 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:02.893472 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fa8ef-18ce-443d-b8e3-1fd238e4cc59 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:02.904522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fa8ef-18ce-443d-b8e3-1fd238e4cc59 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:02.977399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:02.981143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:02.981143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:02.981143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] PATH_INFO: `/v3/projects/b2e8252d092e4087bf47db45b137afc2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:02.984677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e8252d092e4087bf47db45b137afc2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:02.986224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:03.033057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:03.033057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.046006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10419195-1a77-478c-a13a-548a8911bc0f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:03.050908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e8252d092e4087bf47db45b137afc2', 'request_id': 'req-ed70afee-5bb5-4116-a229-b18f74f4ecca'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.056964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e8252d092e4087bf47db45b137afc2', 'request_id': 'req-ed70afee-5bb5-4116-a229-b18f74f4ecca'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.064076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e8252d092e4087bf47db45b137afc2', 'request_id': 'req-ed70afee-5bb5-4116-a229-b18f74f4ecca'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.082343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] The token cache is being invalidate because project b2e8252d092e4087bf47db45b137afc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:03.082634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.083610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.085183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1032/2044] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:02 2022] DELETE /identity/v3/projects/b2e8252d092e4087bf47db45b137afc2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:03.089057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:03.090417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.111755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:03.112675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.112959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.113261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] PATH_INFO: `/v3/users/f36fd12e187d495bae90c61d17227ac3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.114382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f36fd12e187d495bae90c61d17227ac3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:03.116900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:03.156556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70afee-5bb5-4116-a229-b18f74f4ecca admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:03.161089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:03.161733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.162005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.162796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] PATH_INFO: `/v3/users/9c61a19732ea4c32a120b4f4d9f35f7e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.163904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c61a19732ea4c32a120b4f4d9f35f7e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:03.164528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f36fd12e187d495bae90c61d17227ac3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.165850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:03.171679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:03.172309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f36fd12e187d495bae90c61d17227ac3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.173071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f36fd12e187d495bae90c61d17227ac3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.181719 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f36fd12e187d495bae90c61d17227ac3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.197449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15fac23f-bc64-412c-baff-af313c57a921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f36fd12e187d495bae90c61d17227ac3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.200115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:03.211074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1013/2045] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:03 2022] DELETE /identity/v3/users/f36fd12e187d495bae90c61d17227ac3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:03.215626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765c9c6-7b7d-4d2b-b3e6-df9c4e0e06c6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.215902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765c9c6-7b7d-4d2b-b3e6-df9c4e0e06c6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.216326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765c9c6-7b7d-4d2b-b3e6-df9c4e0e06c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.218755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c61a19732ea4c32a120b4f4d9f35f7e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.225890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c61a19732ea4c32a120b4f4d9f35f7e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.226510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c61a19732ea4c32a120b4f4d9f35f7e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.236426 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7765c9c6-7b7d-4d2b-b3e6-df9c4e0e06c6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:03.237901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7765c9c6-7b7d-4d2b-b3e6-df9c4e0e06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.238576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c61a19732ea4c32a120b4f4d9f35f7e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.239234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c61a19732ea4c32a120b4f4d9f35f7e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.251311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1033/2046] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:03 2022] DELETE /identity/v3/users/9c61a19732ea4c32a120b4f4d9f35f7e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:03.304892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1014/2047] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:03.314384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:03.316022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.385205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c35b-6783-4951-a69f-6124eca2a28b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:03.390423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20413d-1a60-41d8-bda5-9ee65b972282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:03.391260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20413d-1a60-41d8-bda5-9ee65b972282 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.391390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20413d-1a60-41d8-bda5-9ee65b972282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.391751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20413d-1a60-41d8-bda5-9ee65b972282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.434596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765c9c6-7b7d-4d2b-b3e6-df9c4e0e06c6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:03.436097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7765c9c6-7b7d-4d2b-b3e6-df9c4e0e06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.451733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765c9c6-7b7d-4d2b-b3e6-df9c4e0e06c6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:03.471152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20413d-1a60-41d8-bda5-9ee65b972282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:03.472629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20413d-1a60-41d8-bda5-9ee65b972282 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:03.485181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1034/2048] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:03.519190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:03.519835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.520114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.520385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] PATH_INFO: `/v3/projects/363961cc0c42473c8d92ae17ed4fa667` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.521563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=363961cc0c42473c8d92ae17ed4fa667)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:03.524012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:03.580114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '363961cc0c42473c8d92ae17ed4fa667', 'request_id': 'req-68e4cdaf-db24-4f42-abe2-367635521769'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.605929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '363961cc0c42473c8d92ae17ed4fa667', 'request_id': 'req-68e4cdaf-db24-4f42-abe2-367635521769'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.613596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '363961cc0c42473c8d92ae17ed4fa667', 'request_id': 'req-68e4cdaf-db24-4f42-abe2-367635521769'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.626545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] The token cache is being invalidate because project 363961cc0c42473c8d92ae17ed4fa667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:03.626836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.627473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-68e4cdaf-db24-4f42-abe2-367635521769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.628994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1015/2049] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:03 2022] DELETE /identity/v3/projects/363961cc0c42473c8d92ae17ed4fa667 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:03.634162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e03fb-d368-4319-b1ac-a095f8f442a4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.634537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e03fb-d368-4319-b1ac-a095f8f442a4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.634906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e03fb-d368-4319-b1ac-a095f8f442a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.659450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e64e03fb-d368-4319-b1ac-a095f8f442a4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:03.661612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e64e03fb-d368-4319-b1ac-a095f8f442a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.703673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:03.739915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1035/2050] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:03.749741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c357-a361-4937-9ba7-4e883f85ee5a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.750034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c357-a361-4937-9ba7-4e883f85ee5a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.751105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c357-a361-4937-9ba7-4e883f85ee5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.774507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-dcc7c357-a361-4937-9ba7-4e883f85ee5a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:03.775724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dcc7c357-a361-4937-9ba7-4e883f85ee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.784230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1016/2051] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:03.789074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e03fb-d368-4319-b1ac-a095f8f442a4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:03.800891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e03fb-d368-4319-b1ac-a095f8f442a4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:03.806680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:03.807373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.807637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.807945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.808716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:03.809052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:03.810802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:03.816466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7c357-a361-4937-9ba7-4e883f85ee5a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:03.817901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dcc7c357-a361-4937-9ba7-4e883f85ee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.839308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1036/2052] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:39:03.843938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:03.849815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7c357-a361-4937-9ba7-4e883f85ee5a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:03.858189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:03.858846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.859128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.859403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] PATH_INFO: `/v3/projects/9aa8759e166c497d9fdba8e566fd98ec` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.860611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa8759e166c497d9fdba8e566fd98ec)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:03.863003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:03.870229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f890d-728a-41b2-b638-a9f93df7f31e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:03.876793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee47ec-a1e4-44c5-997f-1e729efc9518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:03.877435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee47ec-a1e4-44c5-997f-1e729efc9518 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.878140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee47ec-a1e4-44c5-997f-1e729efc9518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.878509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee47ec-a1e4-44c5-997f-1e729efc9518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.879137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee47ec-a1e4-44c5-997f-1e729efc9518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:03.881996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee47ec-a1e4-44c5-997f-1e729efc9518 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:03.888179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:03.917510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1037/2053] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:03 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:03.919323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa8759e166c497d9fdba8e566fd98ec', 'request_id': 'req-0b48122a-4a40-480b-add1-ec65aa5e3690'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.921519 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee47ec-a1e4-44c5-997f-1e729efc9518 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:03.926173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa8759e166c497d9fdba8e566fd98ec', 'request_id': 'req-0b48122a-4a40-480b-add1-ec65aa5e3690'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.940335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa8759e166c497d9fdba8e566fd98ec', 'request_id': 'req-0b48122a-4a40-480b-add1-ec65aa5e3690'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.947804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] The token cache is being invalidate because project 9aa8759e166c497d9fdba8e566fd98ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:03.948164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.948862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b48122a-4a40-480b-add1-ec65aa5e3690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:03.950430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1017/2054] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:03 2022] DELETE /identity/v3/projects/9aa8759e166c497d9fdba8e566fd98ec => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:03.955198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c4eeb-9506-4606-a99e-a1482951f7b9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.955473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c4eeb-9506-4606-a99e-a1482951f7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.955749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c4eeb-9506-4606-a99e-a1482951f7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.976678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-be3c4eeb-9506-4606-a99e-a1482951f7b9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:03.978082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-be3c4eeb-9506-4606-a99e-a1482951f7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:03.986428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1018/2055] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:03.994682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737526f7-2692-4b85-b6eb-d8d85a5e7c2f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:03.994990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737526f7-2692-4b85-b6eb-d8d85a5e7c2f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:03.995318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737526f7-2692-4b85-b6eb-d8d85a5e7c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:03.997563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee47ec-a1e4-44c5-997f-1e729efc9518 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.002225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.002826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.003109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.003377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.003996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.006492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.015189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-737526f7-2692-4b85-b6eb-d8d85a5e7c2f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:04.016405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-737526f7-2692-4b85-b6eb-d8d85a5e7c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:04.026077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1019/2056] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:04.029630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1038/2057] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:03 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:04.030453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737526f7-2692-4b85-b6eb-d8d85a5e7c2f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.033311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.034914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:04.040375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737526f7-2692-4b85-b6eb-d8d85a5e7c2f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.045024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.045652 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.045928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.046195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.046810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e826b0-843e-42b2-b8ad-8633ba62c63f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.046918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:04.047106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.048673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.051611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e26e3-27e7-40e1-a60c-4cd03c4ab899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.052208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e26e3-27e7-40e1-a60c-4cd03c4ab899 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.052478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e26e3-27e7-40e1-a60c-4cd03c4ab899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.052747 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e26e3-27e7-40e1-a60c-4cd03c4ab899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.053460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e26e3-27e7-40e1-a60c-4cd03c4ab899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.055056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e26e3-27e7-40e1-a60c-4cd03c4ab899 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.060680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1020/2058] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:39:04.064812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.067644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1039/2059] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:39:04.069199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e26e3-27e7-40e1-a60c-4cd03c4ab899 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.075254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4104a01-3278-4ade-af83-102f28b8b891 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.079184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e26e3-27e7-40e1-a60c-4cd03c4ab899 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.080580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f504e6-3feb-4b24-85e2-a7d60e659c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.080765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f504e6-3feb-4b24-85e2-a7d60e659c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.081037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f504e6-3feb-4b24-85e2-a7d60e659c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.081343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f504e6-3feb-4b24-85e2-a7d60e659c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.081982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f504e6-3feb-4b24-85e2-a7d60e659c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.083860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f504e6-3feb-4b24-85e2-a7d60e659c73 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.085199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad472b0-76e1-4261-9382-819708a3c987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.085808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad472b0-76e1-4261-9382-819708a3c987 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.086084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad472b0-76e1-4261-9382-819708a3c987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.086353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad472b0-76e1-4261-9382-819708a3c987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.087134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad472b0-76e1-4261-9382-819708a3c987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.088729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad472b0-76e1-4261-9382-819708a3c987 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.099558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1040/2060] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:04.103614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad472b0-76e1-4261-9382-819708a3c987 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.115937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1021/2061] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:04 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:04.122144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f504e6-3feb-4b24-85e2-a7d60e659c73 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.123546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad472b0-76e1-4261-9382-819708a3c987 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.184442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85620d1a-2ec8-43bc-99c3-25711dd24dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.185179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620d1a-2ec8-43bc-99c3-25711dd24dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.185557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620d1a-2ec8-43bc-99c3-25711dd24dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.185826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620d1a-2ec8-43bc-99c3-25711dd24dc4 admin admin] PATH_INFO: `/v3/projects/9036bd761a1e42c285cf20965136efe9/users/ae477ee526df42af84bd782747722205/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.188542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f504e6-3feb-4b24-85e2-a7d60e659c73 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.193207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388b5e1-6bb9-4ac6-bcab-93271a4fc739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.194461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388b5e1-6bb9-4ac6-bcab-93271a4fc739 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.194738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388b5e1-6bb9-4ac6-bcab-93271a4fc739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.195008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388b5e1-6bb9-4ac6-bcab-93271a4fc739 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.195626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388b5e1-6bb9-4ac6-bcab-93271a4fc739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.197618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388b5e1-6bb9-4ac6-bcab-93271a4fc739 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.198388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:04.202974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85620d1a-2ec8-43bc-99c3-25711dd24dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9036bd761a1e42c285cf20965136efe9, user_id=ae477ee526df42af84bd782747722205, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.206132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85620d1a-2ec8-43bc-99c3-25711dd24dc4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.219364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1041/2062] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:04 2022] PUT /identity/v3/projects/9036bd761a1e42c285cf20965136efe9/users/ae477ee526df42af84bd782747722205/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:04.220835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1022/2063] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:04 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:04.224835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85620d1a-2ec8-43bc-99c3-25711dd24dc4 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.231958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90462af-4440-4c5f-805a-9fcd824bfd33 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.232243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90462af-4440-4c5f-805a-9fcd824bfd33 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.232512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90462af-4440-4c5f-805a-9fcd824bfd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.262258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d90462af-4440-4c5f-805a-9fcd824bfd33 None None] MFA Rules not processed for user `ae477ee526df42af84bd782747722205`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:04.263533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d90462af-4440-4c5f-805a-9fcd824bfd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:04.293217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85620d1a-2ec8-43bc-99c3-25711dd24dc4 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.298248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ec13-78ca-4bba-b6fe-f610379bb8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.298995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ec13-78ca-4bba-b6fe-f610379bb8fd admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.299270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ec13-78ca-4bba-b6fe-f610379bb8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.299539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ec13-78ca-4bba-b6fe-f610379bb8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.300203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ec13-78ca-4bba-b6fe-f610379bb8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.301859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ec13-78ca-4bba-b6fe-f610379bb8fd admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.303942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:04.316517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1042/2064] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:04.317942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ec13-78ca-4bba-b6fe-f610379bb8fd admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.328376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ec13-78ca-4bba-b6fe-f610379bb8fd admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.333667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1aab6-ad9f-4344-a933-cca6ef53ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.334765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1aab6-ad9f-4344-a933-cca6ef53ad5b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.335172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1aab6-ad9f-4344-a933-cca6ef53ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.335377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1023/2065] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:04.335479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1aab6-ad9f-4344-a933-cca6ef53ad5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.336202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1aab6-ad9f-4344-a933-cca6ef53ad5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.337868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1aab6-ad9f-4344-a933-cca6ef53ad5b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.346351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90462af-4440-4c5f-805a-9fcd824bfd33 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.347700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d90462af-4440-4c5f-805a-9fcd824bfd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:04.351687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1043/2066] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:04.352463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1aab6-ad9f-4344-a933-cca6ef53ad5b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.361913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1aab6-ad9f-4344-a933-cca6ef53ad5b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.366713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03648d7-bd3f-4440-9708-73c203d6b635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.367335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03648d7-bd3f-4440-9708-73c203d6b635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.367610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03648d7-bd3f-4440-9708-73c203d6b635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.367877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03648d7-bd3f-4440-9708-73c203d6b635 admin admin] PATH_INFO: `/v3/projects/ec4f4f1bbe6a43b1a7644938a960a4fd/users/1e33cb45ee9b4b0ea4c6749caf9dc96f/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.394128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03648d7-bd3f-4440-9708-73c203d6b635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4f4f1bbe6a43b1a7644938a960a4fd, user_id=1e33cb45ee9b4b0ea4c6749caf9dc96f, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.397028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03648d7-bd3f-4440-9708-73c203d6b635 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.411720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1044/2067] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:04 2022] PUT /identity/v3/projects/ec4f4f1bbe6a43b1a7644938a960a4fd/users/1e33cb45ee9b4b0ea4c6749caf9dc96f/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:04.414246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90462af-4440-4c5f-805a-9fcd824bfd33 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.423814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d421b6c-dcb2-4ca9-8089-39ca58dcca13 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.424099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d421b6c-dcb2-4ca9-8089-39ca58dcca13 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.424364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d421b6c-dcb2-4ca9-8089-39ca58dcca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.450846 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0d421b6c-dcb2-4ca9-8089-39ca58dcca13 None None] MFA Rules not processed for user `1e33cb45ee9b4b0ea4c6749caf9dc96f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:04.452070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0d421b6c-dcb2-4ca9-8089-39ca58dcca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:04.467780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1327a-af72-4429-89d6-a003ced3e9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.468411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1327a-af72-4429-89d6-a003ced3e9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.468687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1327a-af72-4429-89d6-a003ced3e9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.468956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1327a-af72-4429-89d6-a003ced3e9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.527522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1045/2068] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:04.535087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1327a-af72-4429-89d6-a003ced3e9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.536585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1327a-af72-4429-89d6-a003ced3e9e6 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.538494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d421b6c-dcb2-4ca9-8089-39ca58dcca13 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:04.540186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0d421b6c-dcb2-4ca9-8089-39ca58dcca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:04.549571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1024/2069] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:39:04.588937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:04.617857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d421b6c-dcb2-4ca9-8089-39ca58dcca13 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:04.623299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9c53b-9391-4779-84ae-22974c607c96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:04.623898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9c53b-9391-4779-84ae-22974c607c96 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:04.624177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9c53b-9391-4779-84ae-22974c607c96 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:04.624443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9c53b-9391-4779-84ae-22974c607c96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:04.632859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9c53b-9391-4779-84ae-22974c607c96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:04.634953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9c53b-9391-4779-84ae-22974c607c96 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:04.644054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1046/2070] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:39:10.610985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1327a-af72-4429-89d6-a003ced3e9e6 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:10.659014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:10.674023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1327a-af72-4429-89d6-a003ced3e9e6 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:10.678575 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:10.679192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:10.679464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:10.679728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] PATH_INFO: `/v3/users/1e33cb45ee9b4b0ea4c6749caf9dc96f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:10.680822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e33cb45ee9b4b0ea4c6749caf9dc96f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:10.682763 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:10.728299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e33cb45ee9b4b0ea4c6749caf9dc96f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:10.735582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e33cb45ee9b4b0ea4c6749caf9dc96f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:10.736247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e33cb45ee9b4b0ea4c6749caf9dc96f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:10.743479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e33cb45ee9b4b0ea4c6749caf9dc96f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:10.755589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e33cb45ee9b4b0ea4c6749caf9dc96f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:10.762211 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1025/2071] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:10 2022] DELETE /identity/v3/users/1e33cb45ee9b4b0ea4c6749caf9dc96f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:10.766604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136874d-0a77-440f-b485-435b876182d1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:10.767061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136874d-0a77-440f-b485-435b876182d1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:10.767170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136874d-0a77-440f-b485-435b876182d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:10.785657 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d136874d-0a77-440f-b485-435b876182d1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:10.786827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d136874d-0a77-440f-b485-435b876182d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:10.845072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1047/2072] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:10.856341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:10.858008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:10.921350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af292b-66f2-4166-ac14-c4008ec641d8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:10.924993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e75df-5dad-445f-8f68-0989cb5a070c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:10.925598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e75df-5dad-445f-8f68-0989cb5a070c service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:10.925897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e75df-5dad-445f-8f68-0989cb5a070c service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:10.926166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e75df-5dad-445f-8f68-0989cb5a070c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:10.935643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e75df-5dad-445f-8f68-0989cb5a070c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:10.937064 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e75df-5dad-445f-8f68-0989cb5a070c service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:10.946579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1026/2073] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:11.236871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136874d-0a77-440f-b485-435b876182d1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:11.238663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d136874d-0a77-440f-b485-435b876182d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:11.255537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136874d-0a77-440f-b485-435b876182d1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:11.261507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:11.262141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.262426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.262665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] PATH_INFO: `/v3/projects/ec4f4f1bbe6a43b1a7644938a960a4fd` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.263766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4f4f1bbe6a43b1a7644938a960a4fd)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:11.265711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:11.316425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4f4f1bbe6a43b1a7644938a960a4fd', 'request_id': 'req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:11.322956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4f4f1bbe6a43b1a7644938a960a4fd', 'request_id': 'req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:11.329461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4f4f1bbe6a43b1a7644938a960a4fd', 'request_id': 'req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:11.343354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] The token cache is being invalidate because project ec4f4f1bbe6a43b1a7644938a960a4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:11.343638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:11.344254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3c3d90a0-0ce9-40c9-ab46-79ad5e2b625d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:11.345895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1048/2074] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:11 2022] DELETE /identity/v3/projects/ec4f4f1bbe6a43b1a7644938a960a4fd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:11.350783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247734b-a976-4c78-b58c-dc78c61e82ab None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.351084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247734b-a976-4c78-b58c-dc78c61e82ab None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.351353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247734b-a976-4c78-b58c-dc78c61e82ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.370481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a247734b-a976-4c78-b58c-dc78c61e82ab None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:11.371676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a247734b-a976-4c78-b58c-dc78c61e82ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:11.428655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1027/2075] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:11.436576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5bfaa-0ff7-4e7b-8744-a45d2b50df5b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.436856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5bfaa-0ff7-4e7b-8744-a45d2b50df5b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.437129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5bfaa-0ff7-4e7b-8744-a45d2b50df5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.456588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a3e5bfaa-0ff7-4e7b-8744-a45d2b50df5b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:11.457816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a3e5bfaa-0ff7-4e7b-8744-a45d2b50df5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:11.466291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1049/2076] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:11.470435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247734b-a976-4c78-b58c-dc78c61e82ab None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:11.479880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247734b-a976-4c78-b58c-dc78c61e82ab None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:11.484460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:11.485045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.485345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.485614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.486248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:11.486541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:11.488379 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:11.495284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:11.504732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1028/2077] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:11.506488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5bfaa-0ff7-4e7b-8744-a45d2b50df5b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:11.520583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5bfaa-0ff7-4e7b-8744-a45d2b50df5b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:11.525947 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a65539-3507-4ece-accc-d77eb2144c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:11.526538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a65539-3507-4ece-accc-d77eb2144c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.526809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a65539-3507-4ece-accc-d77eb2144c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.527073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a65539-3507-4ece-accc-d77eb2144c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.527685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a65539-3507-4ece-accc-d77eb2144c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:11.529647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a65539-3507-4ece-accc-d77eb2144c0d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:11.559916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1050/2078] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:11 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:11.564036 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:11.573880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29f4d6-6a69-4399-8ec9-15ce4a5aa17c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:11.629085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dbc71-ba1c-45fb-91b1-6dc6ac972875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:11.629757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dbc71-ba1c-45fb-91b1-6dc6ac972875 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.629979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dbc71-ba1c-45fb-91b1-6dc6ac972875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.630243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dbc71-ba1c-45fb-91b1-6dc6ac972875 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.630877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866dbc71-ba1c-45fb-91b1-6dc6ac972875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:11.632767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866dbc71-ba1c-45fb-91b1-6dc6ac972875 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:11.653560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1029/2079] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:11 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:11.657775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a65539-3507-4ece-accc-d77eb2144c0d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:11.668590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a65539-3507-4ece-accc-d77eb2144c0d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:11.673419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8100c7-9d30-41c8-b507-b2b458187fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:11.674013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8100c7-9d30-41c8-b507-b2b458187fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.674321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8100c7-9d30-41c8-b507-b2b458187fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.674590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8100c7-9d30-41c8-b507-b2b458187fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.675235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8100c7-9d30-41c8-b507-b2b458187fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:11.676824 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8100c7-9d30-41c8-b507-b2b458187fd1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:11.686612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1051/2080] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:11.690354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dbc71-ba1c-45fb-91b1-6dc6ac972875 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:11.700166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dbc71-ba1c-45fb-91b1-6dc6ac972875 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:11.704738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81bf87-b3fa-41cf-bbf7-1ac853051bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:11.705423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bf87-b3fa-41cf-bbf7-1ac853051bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.705702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bf87-b3fa-41cf-bbf7-1ac853051bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.705967 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bf87-b3fa-41cf-bbf7-1ac853051bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.706593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81bf87-b3fa-41cf-bbf7-1ac853051bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:11.708156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81bf87-b3fa-41cf-bbf7-1ac853051bf8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:11.718892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1030/2081] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:11.722921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8100c7-9d30-41c8-b507-b2b458187fd1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:11.733385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8100c7-9d30-41c8-b507-b2b458187fd1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:11.738326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6096ab-e8be-4600-8c82-906814095503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:11.738952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6096ab-e8be-4600-8c82-906814095503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.739228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6096ab-e8be-4600-8c82-906814095503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.739499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6096ab-e8be-4600-8c82-906814095503 admin admin] PATH_INFO: `/v3/projects/2fd361f27e6143209031a9e8461026b4/users/e9268d39d479420b94ff479bf4027526/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.748165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:11.754988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6096ab-e8be-4600-8c82-906814095503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd361f27e6143209031a9e8461026b4, user_id=e9268d39d479420b94ff479bf4027526, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:11.758811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6096ab-e8be-4600-8c82-906814095503 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:11.772060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1052/2082] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:11 2022] PUT /identity/v3/projects/2fd361f27e6143209031a9e8461026b4/users/e9268d39d479420b94ff479bf4027526/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:11.784554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312d37-feef-4959-8ed4-bf41c1348720 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.784832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312d37-feef-4959-8ed4-bf41c1348720 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.785099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312d37-feef-4959-8ed4-bf41c1348720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.813902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-31312d37-feef-4959-8ed4-bf41c1348720 None None] MFA Rules not processed for user `e9268d39d479420b94ff479bf4027526`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:11.815104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-31312d37-feef-4959-8ed4-bf41c1348720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:11.888956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1031/2083] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:11.893419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6096ab-e8be-4600-8c82-906814095503 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:11.962599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6096ab-e8be-4600-8c82-906814095503 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:11.967303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470a813-6fba-40aa-8d1d-4c99b07c01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:11.967896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470a813-6fba-40aa-8d1d-4c99b07c01d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:11.968168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470a813-6fba-40aa-8d1d-4c99b07c01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:11.968434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470a813-6fba-40aa-8d1d-4c99b07c01d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:11.969041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470a813-6fba-40aa-8d1d-4c99b07c01d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:11.970956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470a813-6fba-40aa-8d1d-4c99b07c01d6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:11.998795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1053/2084] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:11 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:12.002834 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31312d37-feef-4959-8ed4-bf41c1348720 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:12.013846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31312d37-feef-4959-8ed4-bf41c1348720 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:12.069585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24ad28-2e04-4e7b-9c0a-c1e8b4eee2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:12.070195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24ad28-2e04-4e7b-9c0a-c1e8b4eee2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.070470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24ad28-2e04-4e7b-9c0a-c1e8b4eee2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.070737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24ad28-2e04-4e7b-9c0a-c1e8b4eee2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.071359 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24ad28-2e04-4e7b-9c0a-c1e8b4eee2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:12.073276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24ad28-2e04-4e7b-9c0a-c1e8b4eee2d6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:12.084461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:12.098298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1032/2085] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:12 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:12.102854 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470a813-6fba-40aa-8d1d-4c99b07c01d6 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:12.118046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470a813-6fba-40aa-8d1d-4c99b07c01d6 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:12.122677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f2ead-f47f-4068-9377-b0e11d5b8206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:12.123308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f2ead-f47f-4068-9377-b0e11d5b8206 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.123581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f2ead-f47f-4068-9377-b0e11d5b8206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.123847 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f2ead-f47f-4068-9377-b0e11d5b8206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.124487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f2ead-f47f-4068-9377-b0e11d5b8206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:12.126094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f2ead-f47f-4068-9377-b0e11d5b8206 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:12.137183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1054/2086] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:39:12.140922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24ad28-2e04-4e7b-9c0a-c1e8b4eee2d6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:12.150893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24ad28-2e04-4e7b-9c0a-c1e8b4eee2d6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:12.155706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eabd6a-6a75-40ce-9062-9e2b17673307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:12.156347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eabd6a-6a75-40ce-9062-9e2b17673307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.156620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eabd6a-6a75-40ce-9062-9e2b17673307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.156885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eabd6a-6a75-40ce-9062-9e2b17673307 admin admin] PATH_INFO: `/v3/projects/896faedd706a4c07992706deabf6e85c/users/8bee8c049f77405894a478d3a366bac9/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.172112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eabd6a-6a75-40ce-9062-9e2b17673307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896faedd706a4c07992706deabf6e85c, user_id=8bee8c049f77405894a478d3a366bac9, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:12.174808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eabd6a-6a75-40ce-9062-9e2b17673307 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:12.189960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1033/2087] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:12 2022] PUT /identity/v3/projects/896faedd706a4c07992706deabf6e85c/users/8bee8c049f77405894a478d3a366bac9/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:12.194461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f2ead-f47f-4068-9377-b0e11d5b8206 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:12.204621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f2ead-f47f-4068-9377-b0e11d5b8206 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:12.263494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bcd6d-c6f1-409d-aefb-ebaa9cb76d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:12.264123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bcd6d-c6f1-409d-aefb-ebaa9cb76d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.264400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bcd6d-c6f1-409d-aefb-ebaa9cb76d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.264672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bcd6d-c6f1-409d-aefb-ebaa9cb76d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.265337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bcd6d-c6f1-409d-aefb-ebaa9cb76d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:12.266962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bcd6d-c6f1-409d-aefb-ebaa9cb76d5f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:12.277439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1055/2088] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:12.281402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eabd6a-6a75-40ce-9062-9e2b17673307 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:12.291965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eabd6a-6a75-40ce-9062-9e2b17673307 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:12.296674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:12.297378 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.297659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.297932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] PATH_INFO: `/v3/projects/896faedd706a4c07992706deabf6e85c/users/8bee8c049f77405894a478d3a366bac9/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.299851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896faedd706a4c07992706deabf6e85c, user_id=8bee8c049f77405894a478d3a366bac9, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:12.302540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:12.312649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1034/2089] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:12 2022] PUT /identity/v3/projects/896faedd706a4c07992706deabf6e85c/users/8bee8c049f77405894a478d3a366bac9/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:12.323097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a416ce-50c3-45a9-9777-3d3807eabcb8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.323372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a416ce-50c3-45a9-9777-3d3807eabcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.323638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a416ce-50c3-45a9-9777-3d3807eabcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.350479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-34a416ce-50c3-45a9-9777-3d3807eabcb8 None None] MFA Rules not processed for user `8bee8c049f77405894a478d3a366bac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:12.351654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-34a416ce-50c3-45a9-9777-3d3807eabcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:12.397701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:12.421746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1056/2090] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:12.433034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:12.434766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:12.493441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1681cc-e45f-4a36-9935-f5ef9dce7f70 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:12.497030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a6e2-cb0d-46e6-9705-4f3ea1dc5cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:12.497768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a6e2-cb0d-46e6-9705-4f3ea1dc5cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.498042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a6e2-cb0d-46e6-9705-4f3ea1dc5cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.498308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a6e2-cb0d-46e6-9705-4f3ea1dc5cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.507835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a6e2-cb0d-46e6-9705-4f3ea1dc5cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:12.509265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a6e2-cb0d-46e6-9705-4f3ea1dc5cf7 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:12.518381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1035/2091] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:12.547868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a416ce-50c3-45a9-9777-3d3807eabcb8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:12.607386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a416ce-50c3-45a9-9777-3d3807eabcb8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:12.611934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:12.612833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.613106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.613403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] PATH_INFO: `/v3/users/e9268d39d479420b94ff479bf4027526` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.614506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9268d39d479420b94ff479bf4027526)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:12.616464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:12.660113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9268d39d479420b94ff479bf4027526'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:12.669193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9268d39d479420b94ff479bf4027526'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:12.670005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9268d39d479420b94ff479bf4027526'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:12.681984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9268d39d479420b94ff479bf4027526'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:12.682623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9268d39d479420b94ff479bf4027526'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:12.694702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1057/2092] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:12 2022] DELETE /identity/v3/users/e9268d39d479420b94ff479bf4027526 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:12.699330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bf83f-b490-4504-ac9e-9d7e65ab6592 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.699846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bf83f-b490-4504-ac9e-9d7e65ab6592 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.699846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bf83f-b490-4504-ac9e-9d7e65ab6592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.718576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c26bf83f-b490-4504-ac9e-9d7e65ab6592 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:12.719796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c26bf83f-b490-4504-ac9e-9d7e65ab6592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:12.777845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1036/2093] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:12.787917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:12.789342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:12.845047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb5fe6-2a3d-4d75-acbf-13ddfe0ea9e8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:12.848660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac46b5-c3e6-4724-bf30-77dab112fa7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:12.849287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac46b5-c3e6-4724-bf30-77dab112fa7f service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:12.850069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac46b5-c3e6-4724-bf30-77dab112fa7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:12.850362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac46b5-c3e6-4724-bf30-77dab112fa7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:12.859091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac46b5-c3e6-4724-bf30-77dab112fa7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:12.860531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac46b5-c3e6-4724-bf30-77dab112fa7f service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:12.870728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1058/2094] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:39:13.130269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bf83f-b490-4504-ac9e-9d7e65ab6592 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:13.132576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c26bf83f-b490-4504-ac9e-9d7e65ab6592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:13.144828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bf83f-b490-4504-ac9e-9d7e65ab6592 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:13.151240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:13.151954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:13.152194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:13.152465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] PATH_INFO: `/v3/projects/2fd361f27e6143209031a9e8461026b4` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:13.153678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd361f27e6143209031a9e8461026b4)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:13.155627 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:13.211307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd361f27e6143209031a9e8461026b4', 'request_id': 'req-41120716-9648-4ded-84d5-06f5db3b6303'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.216054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:13.218687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd361f27e6143209031a9e8461026b4', 'request_id': 'req-41120716-9648-4ded-84d5-06f5db3b6303'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.229996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd361f27e6143209031a9e8461026b4', 'request_id': 'req-41120716-9648-4ded-84d5-06f5db3b6303'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.255277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] The token cache is being invalidate because project 2fd361f27e6143209031a9e8461026b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:13.256418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.257062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.258901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1037/2095] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:13 2022] DELETE /identity/v3/projects/2fd361f27e6143209031a9e8461026b4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:13.263248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac46b5-c3e6-4724-bf30-77dab112fa7f service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:13.271884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:13.278411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac46b5-c3e6-4724-bf30-77dab112fa7f service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:13.340145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:13.340769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:13.341042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:13.341347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] PATH_INFO: `/v3/users/8bee8c049f77405894a478d3a366bac9` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:13.342497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bee8c049f77405894a478d3a366bac9)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:13.344412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:13.391484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bee8c049f77405894a478d3a366bac9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.399440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bee8c049f77405894a478d3a366bac9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.400066 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bee8c049f77405894a478d3a366bac9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.413119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bee8c049f77405894a478d3a366bac9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.413673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-711e0d3e-77f0-4d9b-a75e-c9872c5875d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bee8c049f77405894a478d3a366bac9'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.426535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1059/2096] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:13 2022] DELETE /identity/v3/users/8bee8c049f77405894a478d3a366bac9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:13.690077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:13.691734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:13.705859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41120716-9648-4ded-84d5-06f5db3b6303 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:13.763873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:13.764437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:13.764709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:13.764976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] PATH_INFO: `/v3/projects/896faedd706a4c07992706deabf6e85c` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:13.766090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=896faedd706a4c07992706deabf6e85c)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:13.768060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:13.812517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896faedd706a4c07992706deabf6e85c', 'request_id': 'req-6ee8099a-8789-4e16-b3c9-854657368abd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.817869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896faedd706a4c07992706deabf6e85c', 'request_id': 'req-6ee8099a-8789-4e16-b3c9-854657368abd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.825145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896faedd706a4c07992706deabf6e85c', 'request_id': 'req-6ee8099a-8789-4e16-b3c9-854657368abd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.837848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] The token cache is being invalidate because project 896faedd706a4c07992706deabf6e85c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:13.838131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.838756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ee8099a-8789-4e16-b3c9-854657368abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:13.840190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1038/2097] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:13 2022] DELETE /identity/v3/projects/896faedd706a4c07992706deabf6e85c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:13.845080 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec8909-bfe8-4c4a-94ee-2193f7a52da6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:13.845614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec8909-bfe8-4c4a-94ee-2193f7a52da6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:13.845717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec8909-bfe8-4c4a-94ee-2193f7a52da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:13.865644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-32ec8909-bfe8-4c4a-94ee-2193f7a52da6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:13.866831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-32ec8909-bfe8-4c4a-94ee-2193f7a52da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:13.924409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1060/2098] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:13.932510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b979b4-c856-456b-9f0d-a4f17de24687 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:13.932788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b979b4-c856-456b-9f0d-a4f17de24687 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:13.933076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b979b4-c856-456b-9f0d-a4f17de24687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:13.952813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-47b979b4-c856-456b-9f0d-a4f17de24687 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:13.954038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-47b979b4-c856-456b-9f0d-a4f17de24687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:13.961836 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1039/2099] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:13.965991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec8909-bfe8-4c4a-94ee-2193f7a52da6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:13.975556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec8909-bfe8-4c4a-94ee-2193f7a52da6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:13.980267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:13.980862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:13.981140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:13.981456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:13.982123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:13.982419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:13.984014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:13.996244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1061/2100] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:39:14.000276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b979b4-c856-456b-9f0d-a4f17de24687 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.009851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b979b4-c856-456b-9f0d-a4f17de24687 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.014356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adbbdc-d945-495b-a4a1-ace9914e6b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.015027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adbbdc-d945-495b-a4a1-ace9914e6b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.015303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adbbdc-d945-495b-a4a1-ace9914e6b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.015568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adbbdc-d945-495b-a4a1-ace9914e6b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.016187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adbbdc-d945-495b-a4a1-ace9914e6b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.018157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adbbdc-d945-495b-a4a1-ace9914e6b26 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.048105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1040/2101] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:13 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:14.049376 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.060697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583ddef-d4f8-4704-aa50-6c1b7a021865 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.069067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:14.118047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd27a6-8b10-4504-8374-71308c6d6977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.118662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd27a6-8b10-4504-8374-71308c6d6977 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.118936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd27a6-8b10-4504-8374-71308c6d6977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.119200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd27a6-8b10-4504-8374-71308c6d6977 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.119810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd27a6-8b10-4504-8374-71308c6d6977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.121762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd27a6-8b10-4504-8374-71308c6d6977 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.142754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1062/2102] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:14 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:14.146843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adbbdc-d945-495b-a4a1-ace9914e6b26 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.156940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adbbdc-d945-495b-a4a1-ace9914e6b26 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.161570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2986-956a-4d30-aa45-52880f1e8358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.162166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2986-956a-4d30-aa45-52880f1e8358 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.162444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2986-956a-4d30-aa45-52880f1e8358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.162715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2986-956a-4d30-aa45-52880f1e8358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.163358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a2986-956a-4d30-aa45-52880f1e8358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.164971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a2986-956a-4d30-aa45-52880f1e8358 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.174634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1041/2103] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:14.178757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd27a6-8b10-4504-8374-71308c6d6977 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.188548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd27a6-8b10-4504-8374-71308c6d6977 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.193144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1a12e-e935-482c-86de-efa8bfa1c451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.193816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1a12e-e935-482c-86de-efa8bfa1c451 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.194071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1a12e-e935-482c-86de-efa8bfa1c451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.194340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1a12e-e935-482c-86de-efa8bfa1c451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.195001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1a12e-e935-482c-86de-efa8bfa1c451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.196593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1a12e-e935-482c-86de-efa8bfa1c451 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.207042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1063/2104] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:14.211622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2986-956a-4d30-aa45-52880f1e8358 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.221467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2986-956a-4d30-aa45-52880f1e8358 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.226111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9080e62-4eb8-4901-b0fa-747d06d916d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.226757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9080e62-4eb8-4901-b0fa-747d06d916d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.227034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9080e62-4eb8-4901-b0fa-747d06d916d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.227303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9080e62-4eb8-4901-b0fa-747d06d916d4 admin admin] PATH_INFO: `/v3/projects/e7a7f48931c040d5b6163a2fb0b4ef6b/users/2d4b600f47a94c799a11b39fe8116f60/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.242869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9080e62-4eb8-4901-b0fa-747d06d916d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a7f48931c040d5b6163a2fb0b4ef6b, user_id=2d4b600f47a94c799a11b39fe8116f60, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.245563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9080e62-4eb8-4901-b0fa-747d06d916d4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.260464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1042/2105] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:14 2022] PUT /identity/v3/projects/e7a7f48931c040d5b6163a2fb0b4ef6b/users/2d4b600f47a94c799a11b39fe8116f60/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:14.271668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a074673-8453-4864-ae08-4194bcdade70 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.271994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a074673-8453-4864-ae08-4194bcdade70 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.272269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a074673-8453-4864-ae08-4194bcdade70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.301730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6a074673-8453-4864-ae08-4194bcdade70 None None] MFA Rules not processed for user `2d4b600f47a94c799a11b39fe8116f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:14.303540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6a074673-8453-4864-ae08-4194bcdade70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:14.365901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1064/2106] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:14.370190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9080e62-4eb8-4901-b0fa-747d06d916d4 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.382931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:14.435708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9080e62-4eb8-4901-b0fa-747d06d916d4 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.440991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ba5e9-53b1-4cb7-ad61-ec1974826f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.441648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ba5e9-53b1-4cb7-ad61-ec1974826f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.441924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ba5e9-53b1-4cb7-ad61-ec1974826f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.442190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ba5e9-53b1-4cb7-ad61-ec1974826f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.442814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ba5e9-53b1-4cb7-ad61-ec1974826f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.445251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ba5e9-53b1-4cb7-ad61-ec1974826f2e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.493814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1043/2107] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:14 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:14.497455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a074673-8453-4864-ae08-4194bcdade70 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.511806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a074673-8453-4864-ae08-4194bcdade70 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.574340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba1407-7568-45b2-8ebd-0956038b0e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.575046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1407-7568-45b2-8ebd-0956038b0e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.575334 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1407-7568-45b2-8ebd-0956038b0e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.575835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1407-7568-45b2-8ebd-0956038b0e6a admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.576471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba1407-7568-45b2-8ebd-0956038b0e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.578422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba1407-7568-45b2-8ebd-0956038b0e6a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.611678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1065/2108] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:14 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:14.615963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ba5e9-53b1-4cb7-ad61-ec1974826f2e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.639453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ba5e9-53b1-4cb7-ad61-ec1974826f2e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.644397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf5f8-4eaa-4405-aa2d-47f133226715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.645001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf5f8-4eaa-4405-aa2d-47f133226715 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.645318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf5f8-4eaa-4405-aa2d-47f133226715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.645587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf5f8-4eaa-4405-aa2d-47f133226715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.646328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcf5f8-4eaa-4405-aa2d-47f133226715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.648489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcf5f8-4eaa-4405-aa2d-47f133226715 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.659678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1044/2109] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:39:14.664290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba1407-7568-45b2-8ebd-0956038b0e6a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.675017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba1407-7568-45b2-8ebd-0956038b0e6a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.680220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf7964-abf0-4082-945d-5b06a6fe1010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.680870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7964-abf0-4082-945d-5b06a6fe1010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.681145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7964-abf0-4082-945d-5b06a6fe1010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.681441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7964-abf0-4082-945d-5b06a6fe1010 admin admin] PATH_INFO: `/v3/projects/137c99d1677840378a7c68b9338fb0de/users/4b80105cfd6d45ee8b820b299b1ab39b/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.684698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:14.700037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf7964-abf0-4082-945d-5b06a6fe1010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=137c99d1677840378a7c68b9338fb0de, user_id=4b80105cfd6d45ee8b820b299b1ab39b, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.702867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf7964-abf0-4082-945d-5b06a6fe1010 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.715892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1066/2110] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:14 2022] PUT /identity/v3/projects/137c99d1677840378a7c68b9338fb0de/users/4b80105cfd6d45ee8b820b299b1ab39b/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:14.720047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf5f8-4eaa-4405-aa2d-47f133226715 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.730395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf5f8-4eaa-4405-aa2d-47f133226715 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.787626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c527ae-0440-4afb-8b84-e1a044be51ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.788156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c527ae-0440-4afb-8b84-e1a044be51ec admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.788432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c527ae-0440-4afb-8b84-e1a044be51ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.788698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c527ae-0440-4afb-8b84-e1a044be51ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.789488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c527ae-0440-4afb-8b84-e1a044be51ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.791063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c527ae-0440-4afb-8b84-e1a044be51ec admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.800994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1045/2111] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:14.806398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf7964-abf0-4082-945d-5b06a6fe1010 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.817628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf7964-abf0-4082-945d-5b06a6fe1010 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:14.823318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:14.823976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.824251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.824542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] PATH_INFO: `/v3/projects/137c99d1677840378a7c68b9338fb0de/users/4b80105cfd6d45ee8b820b299b1ab39b/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.826494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=137c99d1677840378a7c68b9338fb0de, user_id=4b80105cfd6d45ee8b820b299b1ab39b, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:14.829330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:14.839325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1067/2112] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:14 2022] PUT /identity/v3/projects/137c99d1677840378a7c68b9338fb0de/users/4b80105cfd6d45ee8b820b299b1ab39b/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:14.849728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e496997-52fa-4126-ba41-a68484cada0a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:14.850010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e496997-52fa-4126-ba41-a68484cada0a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:14.850278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e496997-52fa-4126-ba41-a68484cada0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:14.877954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9e496997-52fa-4126-ba41-a68484cada0a None None] MFA Rules not processed for user `4b80105cfd6d45ee8b820b299b1ab39b`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:14.879133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9e496997-52fa-4126-ba41-a68484cada0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:14.944849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1046/2113] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:14.957072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:14.958826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:15.014095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e0c7-55ee-42a3-9274-bf0b71e5c954 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:15.017693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e74faa-ec86-47a2-81f9-f3f2f4dfd6a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:15.018279 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e74faa-ec86-47a2-81f9-f3f2f4dfd6a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:15.018557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e74faa-ec86-47a2-81f9-f3f2f4dfd6a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:15.018826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e74faa-ec86-47a2-81f9-f3f2f4dfd6a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:15.027510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e74faa-ec86-47a2-81f9-f3f2f4dfd6a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:15.028923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e74faa-ec86-47a2-81f9-f3f2f4dfd6a9 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:15.038676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1068/2114] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:15.350021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e496997-52fa-4126-ba41-a68484cada0a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:15.359164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e496997-52fa-4126-ba41-a68484cada0a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:15.364016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b80105cfd6d45ee8b820b299b1ab39b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137c99d1677840378a7c68b9338fb0de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:15.364612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:15.364890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:15.365161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:15.365855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:15.366158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:15.367756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:15.369191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:15.380345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1047/2115] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:15.384797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e74faa-ec86-47a2-81f9-f3f2f4dfd6a9 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:15.393766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e74faa-ec86-47a2-81f9-f3f2f4dfd6a9 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:15.399118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805980b1-8a83-4963-84dc-882632dc68cd tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b80105cfd6d45ee8b820b299b1ab39b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137c99d1677840378a7c68b9338fb0de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:15.399706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805980b1-8a83-4963-84dc-882632dc68cd tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:15.399984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805980b1-8a83-4963-84dc-882632dc68cd tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:15.400252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805980b1-8a83-4963-84dc-882632dc68cd tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:15.400872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805980b1-8a83-4963-84dc-882632dc68cd tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:15.402797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805980b1-8a83-4963-84dc-882632dc68cd tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:15.432103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1069/2116] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:15 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:15.434331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:15.444831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ecbae-02e4-4660-bb1c-23a6e6e64ae8 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:15.500200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b80105cfd6d45ee8b820b299b1ab39b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137c99d1677840378a7c68b9338fb0de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:15.500806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:15.501083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:15.501387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:15.502047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:15.502344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:15.503924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:15.515415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1048/2117] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:15.551553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805980b1-8a83-4963-84dc-882632dc68cd tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:15.562354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805980b1-8a83-4963-84dc-882632dc68cd tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:15.567113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b80105cfd6d45ee8b820b299b1ab39b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137c99d1677840378a7c68b9338fb0de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:15.567725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:15.568008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:15.568868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] PATH_INFO: `/v3/projects/6e995c860a324b1db58b3c20fbbedf3d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:15.569995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6e995c860a324b1db58b3c20fbbedf3d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:15.571933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:15.625437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e995c860a324b1db58b3c20fbbedf3d', 'request_id': 'req-987ddccc-d546-4bf1-8d1a-294aaa85fa34'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.630872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e995c860a324b1db58b3c20fbbedf3d', 'request_id': 'req-987ddccc-d546-4bf1-8d1a-294aaa85fa34'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.643257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e995c860a324b1db58b3c20fbbedf3d', 'request_id': 'req-987ddccc-d546-4bf1-8d1a-294aaa85fa34'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.660880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] The token cache is being invalidate because project 6e995c860a324b1db58b3c20fbbedf3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:15.661197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.661858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-987ddccc-d546-4bf1-8d1a-294aaa85fa34 tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:15.663939 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1070/2118] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:15 2022] DELETE /identity/v3/projects/6e995c860a324b1db58b3c20fbbedf3d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:15.888019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:15.889856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:15.957672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e0ee7-fe6a-466a-880f-071fae839e6b tempest-VolumeQuotaClassesTest-1606264116 tempest-VolumeQuotaClassesTest-1606264116-project-admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:16.014439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:16.015069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:16.015340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:16.015606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] PATH_INFO: `/v3/users/2d4b600f47a94c799a11b39fe8116f60` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:16.016732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d4b600f47a94c799a11b39fe8116f60)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:16.018696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:16.073338 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:16.082730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d4b600f47a94c799a11b39fe8116f60'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.093150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d4b600f47a94c799a11b39fe8116f60'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.093767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d4b600f47a94c799a11b39fe8116f60'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.102335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d4b600f47a94c799a11b39fe8116f60'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.117254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d4b600f47a94c799a11b39fe8116f60'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.122406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1049/2119] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:15 2022] DELETE /identity/v3/users/2d4b600f47a94c799a11b39fe8116f60 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:16.126797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15196a-7fa1-4c6c-8a7d-bc42894b4406 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:16.127244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15196a-7fa1-4c6c-8a7d-bc42894b4406 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:16.127461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15196a-7fa1-4c6c-8a7d-bc42894b4406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:16.148280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ce15196a-7fa1-4c6c-8a7d-bc42894b4406 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:16.149668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ce15196a-7fa1-4c6c-8a7d-bc42894b4406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:16.181892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:16.233799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1071/2120] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:16.244837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:16.246316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:16.305479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4f51d-aeb0-403b-95a9-d6ccb2d99352 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:16.309490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0644b8-68f3-4f14-81d7-4774061370c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:16.310225 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0644b8-68f3-4f14-81d7-4774061370c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:16.310543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0644b8-68f3-4f14-81d7-4774061370c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:16.310812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0644b8-68f3-4f14-81d7-4774061370c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:16.321089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0644b8-68f3-4f14-81d7-4774061370c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:16.322486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0644b8-68f3-4f14-81d7-4774061370c3 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:16.331832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1050/2121] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:16.606712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15196a-7fa1-4c6c-8a7d-bc42894b4406 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:16.608233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ce15196a-7fa1-4c6c-8a7d-bc42894b4406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:16.621518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15196a-7fa1-4c6c-8a7d-bc42894b4406 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:16.627410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:16.628049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:16.628327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:16.628594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] PATH_INFO: `/v3/projects/e7a7f48931c040d5b6163a2fb0b4ef6b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:16.629759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a7f48931c040d5b6163a2fb0b4ef6b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:16.632365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:16.679300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a7f48931c040d5b6163a2fb0b4ef6b', 'request_id': 'req-91dbb807-ed99-4f3b-a838-d44aa1faaa72'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.684732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a7f48931c040d5b6163a2fb0b4ef6b', 'request_id': 'req-91dbb807-ed99-4f3b-a838-d44aa1faaa72'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.690624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a7f48931c040d5b6163a2fb0b4ef6b', 'request_id': 'req-91dbb807-ed99-4f3b-a838-d44aa1faaa72'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.708721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] The token cache is being invalidate because project e7a7f48931c040d5b6163a2fb0b4ef6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:16.709590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.710203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.711676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1072/2122] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:16 2022] DELETE /identity/v3/projects/e7a7f48931c040d5b6163a2fb0b4ef6b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:16.715660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0644b8-68f3-4f14-81d7-4774061370c3 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:16.725680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0644b8-68f3-4f14-81d7-4774061370c3 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:16.782165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:16.782831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:16.783115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:16.783386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] PATH_INFO: `/v3/users/4b80105cfd6d45ee8b820b299b1ab39b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:16.784497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b80105cfd6d45ee8b820b299b1ab39b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:16.786450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:16.844159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b80105cfd6d45ee8b820b299b1ab39b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.851112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b80105cfd6d45ee8b820b299b1ab39b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.851875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b80105cfd6d45ee8b820b299b1ab39b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.857280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:16.861758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b80105cfd6d45ee8b820b299b1ab39b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.877435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-430fbec8-d432-47a6-b02c-90ffde95ef5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b80105cfd6d45ee8b820b299b1ab39b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:16.884016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1051/2123] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:16 2022] DELETE /identity/v3/users/4b80105cfd6d45ee8b820b299b1ab39b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:17.206086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:17.207894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:17.248613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbb807-ed99-4f3b-a838-d44aa1faaa72 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:17.329046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:17.329711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.329990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.330260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] PATH_INFO: `/v3/projects/137c99d1677840378a7c68b9338fb0de` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.331460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=137c99d1677840378a7c68b9338fb0de)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:17.333433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:17.383892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '137c99d1677840378a7c68b9338fb0de', 'request_id': 'req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:17.389318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '137c99d1677840378a7c68b9338fb0de', 'request_id': 'req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:17.391131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:17.395754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '137c99d1677840378a7c68b9338fb0de', 'request_id': 'req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:17.410291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] The token cache is being invalidate because project 137c99d1677840378a7c68b9338fb0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:17.410574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:17.411190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a87bc2af-6682-45ed-ba9a-aaeb5ed5acc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:17.413170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1073/2124] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:17 2022] DELETE /identity/v3/projects/137c99d1677840378a7c68b9338fb0de => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:17.418079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf8a20-c222-4e8a-8d6e-88e7cc78f746 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.418713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf8a20-c222-4e8a-8d6e-88e7cc78f746 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.418713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf8a20-c222-4e8a-8d6e-88e7cc78f746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.438271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-18cf8a20-c222-4e8a-8d6e-88e7cc78f746 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:17.439448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-18cf8a20-c222-4e8a-8d6e-88e7cc78f746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:17.496145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1052/2125] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:17.504124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33bdeb-f0c1-443e-935e-0b564277a827 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.504406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33bdeb-f0c1-443e-935e-0b564277a827 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.504678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33bdeb-f0c1-443e-935e-0b564277a827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.524879 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ac33bdeb-f0c1-443e-935e-0b564277a827 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:17.526112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ac33bdeb-f0c1-443e-935e-0b564277a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:17.533905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1074/2126] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:17.538161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf8a20-c222-4e8a-8d6e-88e7cc78f746 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:17.547647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf8a20-c222-4e8a-8d6e-88e7cc78f746 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:17.552108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:17.552730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.553008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.553303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.553938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:17.554259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:17.555860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:17.567008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1053/2127] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:17.571495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33bdeb-f0c1-443e-935e-0b564277a827 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:17.580913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33bdeb-f0c1-443e-935e-0b564277a827 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:17.585667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb45b8-6996-41ab-8150-8b76958eac45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:17.586249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb45b8-6996-41ab-8150-8b76958eac45 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.586522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb45b8-6996-41ab-8150-8b76958eac45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.586783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb45b8-6996-41ab-8150-8b76958eac45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.587450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb45b8-6996-41ab-8150-8b76958eac45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:17.589356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb45b8-6996-41ab-8150-8b76958eac45 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:17.618574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1075/2128] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:17 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:17.623384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:17.635063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d26cd-8745-4a30-910f-8872eaf5594e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:17.703246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6383b3d-df2f-43bf-b31a-bd201bdfa260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:17.703864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6383b3d-df2f-43bf-b31a-bd201bdfa260 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.704141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6383b3d-df2f-43bf-b31a-bd201bdfa260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.704409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6383b3d-df2f-43bf-b31a-bd201bdfa260 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.705030 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6383b3d-df2f-43bf-b31a-bd201bdfa260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:17.707140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6383b3d-df2f-43bf-b31a-bd201bdfa260 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:17.728872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1054/2129] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:17 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:17.733116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb45b8-6996-41ab-8150-8b76958eac45 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:17.744572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb45b8-6996-41ab-8150-8b76958eac45 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:17.749315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55f202-fb4b-4380-b995-7116e831c5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:17.750019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f202-fb4b-4380-b995-7116e831c5fc admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.750295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f202-fb4b-4380-b995-7116e831c5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.750564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f202-fb4b-4380-b995-7116e831c5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.751308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55f202-fb4b-4380-b995-7116e831c5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:17.752903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55f202-fb4b-4380-b995-7116e831c5fc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:17.763415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1076/2130] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:17.767443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6383b3d-df2f-43bf-b31a-bd201bdfa260 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:17.777910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6383b3d-df2f-43bf-b31a-bd201bdfa260 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:17.782424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a7f42-485d-4afb-9678-3bcdd1d93363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:17.783018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a7f42-485d-4afb-9678-3bcdd1d93363 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.783281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a7f42-485d-4afb-9678-3bcdd1d93363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.783544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a7f42-485d-4afb-9678-3bcdd1d93363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.784171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a7f42-485d-4afb-9678-3bcdd1d93363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:17.785771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a7f42-485d-4afb-9678-3bcdd1d93363 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:17.799320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1055/2131] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:17.800306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55f202-fb4b-4380-b995-7116e831c5fc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:17.810851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55f202-fb4b-4380-b995-7116e831c5fc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:17.815537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663ee5f-8d54-4814-a7fd-3b4ea3651fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:17.816172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663ee5f-8d54-4814-a7fd-3b4ea3651fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.816447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663ee5f-8d54-4814-a7fd-3b4ea3651fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.816718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663ee5f-8d54-4814-a7fd-3b4ea3651fa9 admin admin] PATH_INFO: `/v3/projects/02952f4070d940febe8112379639d751/users/60ceb9f720f54a02a95fdbbc93d2396b/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.830865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663ee5f-8d54-4814-a7fd-3b4ea3651fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02952f4070d940febe8112379639d751, user_id=60ceb9f720f54a02a95fdbbc93d2396b, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:17.833560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663ee5f-8d54-4814-a7fd-3b4ea3651fa9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:17.846462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1077/2132] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:17 2022] PUT /identity/v3/projects/02952f4070d940febe8112379639d751/users/60ceb9f720f54a02a95fdbbc93d2396b/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:17.858108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0588a-52c9-4829-b1f7-86af7fb3daa8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:17.858386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0588a-52c9-4829-b1f7-86af7fb3daa8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:17.858653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0588a-52c9-4829-b1f7-86af7fb3daa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:17.886270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6ef0588a-52c9-4829-b1f7-86af7fb3daa8 None None] MFA Rules not processed for user `60ceb9f720f54a02a95fdbbc93d2396b`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:17.887380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6ef0588a-52c9-4829-b1f7-86af7fb3daa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:17.918355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:17.953503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1056/2133] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:17.957971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663ee5f-8d54-4814-a7fd-3b4ea3651fa9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:18.023247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663ee5f-8d54-4814-a7fd-3b4ea3651fa9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:18.028441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419981c8-7337-4d82-bfd2-5b8acf433d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:18.029292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419981c8-7337-4d82-bfd2-5b8acf433d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.029591 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419981c8-7337-4d82-bfd2-5b8acf433d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.029887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419981c8-7337-4d82-bfd2-5b8acf433d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.030626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419981c8-7337-4d82-bfd2-5b8acf433d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:18.032738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419981c8-7337-4d82-bfd2-5b8acf433d5f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:18.064620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1078/2134] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:17 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:18.065779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0588a-52c9-4829-b1f7-86af7fb3daa8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:18.077028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0588a-52c9-4829-b1f7-86af7fb3daa8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:18.139933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62777120-38e4-4381-8e6d-906489360e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:18.140471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777120-38e4-4381-8e6d-906489360e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.140748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777120-38e4-4381-8e6d-906489360e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.141015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777120-38e4-4381-8e6d-906489360e57 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.141675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62777120-38e4-4381-8e6d-906489360e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:18.143619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62777120-38e4-4381-8e6d-906489360e57 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:18.165382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1057/2135] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:18 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:18.169403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419981c8-7337-4d82-bfd2-5b8acf433d5f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:18.186411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419981c8-7337-4d82-bfd2-5b8acf433d5f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:18.191167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0fb63-a99e-493d-b361-3020060543ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:18.191798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0fb63-a99e-493d-b361-3020060543ad admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.192073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0fb63-a99e-493d-b361-3020060543ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.192343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0fb63-a99e-493d-b361-3020060543ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.192984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0fb63-a99e-493d-b361-3020060543ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:18.194635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0fb63-a99e-493d-b361-3020060543ad admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:18.206070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1079/2136] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:18.211134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62777120-38e4-4381-8e6d-906489360e57 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:18.221791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62777120-38e4-4381-8e6d-906489360e57 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:18.226995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d4f72-da47-4238-a97b-eaf02ebcbb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:18.227650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d4f72-da47-4238-a97b-eaf02ebcbb72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.227926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d4f72-da47-4238-a97b-eaf02ebcbb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.228194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d4f72-da47-4238-a97b-eaf02ebcbb72 admin admin] PATH_INFO: `/v3/projects/9019dea15e0440dd82ee37ad30e1f816/users/5489fc9126344d2b9aafe940331da31d/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.246831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d4f72-da47-4238-a97b-eaf02ebcbb72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9019dea15e0440dd82ee37ad30e1f816, user_id=5489fc9126344d2b9aafe940331da31d, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:18.249630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d4f72-da47-4238-a97b-eaf02ebcbb72 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:18.270674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1058/2137] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:18 2022] PUT /identity/v3/projects/9019dea15e0440dd82ee37ad30e1f816/users/5489fc9126344d2b9aafe940331da31d/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:18.277170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0fb63-a99e-493d-b361-3020060543ad admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:18.291570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0fb63-a99e-493d-b361-3020060543ad admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:18.376179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628b9a2-1576-4341-bf92-532fe187da05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:18.376882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b9a2-1576-4341-bf92-532fe187da05 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.377253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b9a2-1576-4341-bf92-532fe187da05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.377589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628b9a2-1576-4341-bf92-532fe187da05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.378304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628b9a2-1576-4341-bf92-532fe187da05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:18.382288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628b9a2-1576-4341-bf92-532fe187da05 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:18.383914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:18.399136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1080/2138] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:18.410589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d4f72-da47-4238-a97b-eaf02ebcbb72 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:18.426784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d4f72-da47-4238-a97b-eaf02ebcbb72 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:18.434135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac8224-2465-4130-8bfc-ecd63902807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:18.434856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac8224-2465-4130-8bfc-ecd63902807f admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.435180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac8224-2465-4130-8bfc-ecd63902807f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.435453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac8224-2465-4130-8bfc-ecd63902807f admin admin] PATH_INFO: `/v3/projects/9019dea15e0440dd82ee37ad30e1f816/users/5489fc9126344d2b9aafe940331da31d/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.438020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac8224-2465-4130-8bfc-ecd63902807f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9019dea15e0440dd82ee37ad30e1f816, user_id=5489fc9126344d2b9aafe940331da31d, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:18.441284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac8224-2465-4130-8bfc-ecd63902807f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:18.449516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710de28-b373-4b8b-af68-43c9e5112623 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.449798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710de28-b373-4b8b-af68-43c9e5112623 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.450065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710de28-b373-4b8b-af68-43c9e5112623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.461182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1059/2139] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:18 2022] PUT /identity/v3/projects/9019dea15e0440dd82ee37ad30e1f816/users/5489fc9126344d2b9aafe940331da31d/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:18.471749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab163f-cb42-4f62-a931-e908f6315f25 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.472206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab163f-cb42-4f62-a931-e908f6315f25 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.472436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab163f-cb42-4f62-a931-e908f6315f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.480253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6710de28-b373-4b8b-af68-43c9e5112623 None None] MFA Rules not processed for user `411ff3f0306848ee91108343982806ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:18.482199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6710de28-b373-4b8b-af68-43c9e5112623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:18.521644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d2ab163f-cb42-4f62-a931-e908f6315f25 None None] MFA Rules not processed for user `5489fc9126344d2b9aafe940331da31d`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:18.525313 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d2ab163f-cb42-4f62-a931-e908f6315f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:18.586427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1081/2140] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 10 13:39:18.625097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1060/2141] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:18.680159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710de28-b373-4b8b-af68-43c9e5112623 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:18.682039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6710de28-b373-4b8b-af68-43c9e5112623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:18.751690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710de28-b373-4b8b-af68-43c9e5112623 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:18.755578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:18.756069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.756343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.756612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.775973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:18.777607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:18.802777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1082/2142] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:18.922804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab163f-cb42-4f62-a931-e908f6315f25 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:18.925430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d2ab163f-cb42-4f62-a931-e908f6315f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:18.939450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab163f-cb42-4f62-a931-e908f6315f25 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:18.943982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef11750-b949-4936-a278-993162d7eba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:18.944708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11750-b949-4936-a278-993162d7eba6 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:18.945085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11750-b949-4936-a278-993162d7eba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:18.945492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11750-b949-4936-a278-993162d7eba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:18.951251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:19.014723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef11750-b949-4936-a278-993162d7eba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:19.016187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef11750-b949-4936-a278-993162d7eba6 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:19.033432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1061/2143] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:22.367607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:22.369285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:22.446731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61695a2f-79f6-40d5-8591-da8331cb776a service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:22.452478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:22.453190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:22.453593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:22.453992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] PATH_INFO: `/v3/users/8f49a52c17954d7fa609c65c560fd36c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:22.455292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f49a52c17954d7fa609c65c560fd36c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:22.457452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:22.518517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:22.520426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f49a52c17954d7fa609c65c560fd36c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:22.534559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f49a52c17954d7fa609c65c560fd36c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:22.536078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f49a52c17954d7fa609c65c560fd36c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:22.557000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f49a52c17954d7fa609c65c560fd36c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:22.557931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f49a52c17954d7fa609c65c560fd36c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:22.580897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1083/2144] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:22 2022] DELETE /identity/v3/users/8f49a52c17954d7fa609c65c560fd36c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:22.585849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6434bb-4c57-481f-834a-cc380b1a4ec9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:22.586331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6434bb-4c57-481f-834a-cc380b1a4ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:22.586528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6434bb-4c57-481f-834a-cc380b1a4ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:22.608777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-dd6434bb-4c57-481f-834a-cc380b1a4ec9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:22.610305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dd6434bb-4c57-481f-834a-cc380b1a4ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:22.682740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1062/2145] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:22.693655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:22.695109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:22.787272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624e026-7f56-46f7-9661-fb8352bd891c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:22.792161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d2cef-b98e-474b-9750-a610d5ef6630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:22.792608 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d2cef-b98e-474b-9750-a610d5ef6630 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:22.793114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d2cef-b98e-474b-9750-a610d5ef6630 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:22.793493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d2cef-b98e-474b-9750-a610d5ef6630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:22.807466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d2cef-b98e-474b-9750-a610d5ef6630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:22.809041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d2cef-b98e-474b-9750-a610d5ef6630 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:22.825213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1084/2146] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:39:23.225999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6434bb-4c57-481f-834a-cc380b1a4ec9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:23.227916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dd6434bb-4c57-481f-834a-cc380b1a4ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:23.268946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6434bb-4c57-481f-834a-cc380b1a4ec9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:23.283252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:23.283966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.284303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.285121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] PATH_INFO: `/v3/projects/565ccb02b2d74ca9a5c443aebe4b69db` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.288108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=565ccb02b2d74ca9a5c443aebe4b69db)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:23.290413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:23.368788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565ccb02b2d74ca9a5c443aebe4b69db', 'request_id': 'req-f07d790f-fda6-45e6-a21c-a53b5fc713d1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:23.375962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565ccb02b2d74ca9a5c443aebe4b69db', 'request_id': 'req-f07d790f-fda6-45e6-a21c-a53b5fc713d1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:23.384024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565ccb02b2d74ca9a5c443aebe4b69db', 'request_id': 'req-f07d790f-fda6-45e6-a21c-a53b5fc713d1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:23.404237 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] The token cache is being invalidate because project 565ccb02b2d74ca9a5c443aebe4b69db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:23.405388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:23.406208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f07d790f-fda6-45e6-a21c-a53b5fc713d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:23.408386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1063/2147] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:23 2022] DELETE /identity/v3/projects/565ccb02b2d74ca9a5c443aebe4b69db => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:23.413949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544a1f4-3076-4ccb-8f3d-97420396384b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.414364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544a1f4-3076-4ccb-8f3d-97420396384b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.414512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544a1f4-3076-4ccb-8f3d-97420396384b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.448356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8544a1f4-3076-4ccb-8f3d-97420396384b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:23.449633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8544a1f4-3076-4ccb-8f3d-97420396384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:23.525469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1085/2148] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:23.528488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a45e65-eaf3-4f07-adb1-14a9a6aa96e9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.528963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a45e65-eaf3-4f07-adb1-14a9a6aa96e9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.529087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a45e65-eaf3-4f07-adb1-14a9a6aa96e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.552513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-66a45e65-eaf3-4f07-adb1-14a9a6aa96e9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:23.553767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-66a45e65-eaf3-4f07-adb1-14a9a6aa96e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:23.569148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1064/2149] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:23.573326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544a1f4-3076-4ccb-8f3d-97420396384b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:23.582882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544a1f4-3076-4ccb-8f3d-97420396384b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:23.588250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:23.588839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.589132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.589412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.590047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:23.590347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:23.591941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:23.603410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1086/2150] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:23.607858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a45e65-eaf3-4f07-adb1-14a9a6aa96e9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:23.617546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a45e65-eaf3-4f07-adb1-14a9a6aa96e9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:23.622275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417d7bd-c5b9-476b-a8d3-0e84f0fb0132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:23.622883 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417d7bd-c5b9-476b-a8d3-0e84f0fb0132 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.623162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417d7bd-c5b9-476b-a8d3-0e84f0fb0132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.623436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417d7bd-c5b9-476b-a8d3-0e84f0fb0132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.624055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417d7bd-c5b9-476b-a8d3-0e84f0fb0132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:23.626286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417d7bd-c5b9-476b-a8d3-0e84f0fb0132 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:23.654577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1065/2151] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:23 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:23.659129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:23.668987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab2075-a331-4c70-8a56-c831e5f638a0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:23.719862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:23.732088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab062e59-829e-474c-bb35-dab921ab2b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:23.732730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab062e59-829e-474c-bb35-dab921ab2b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.733009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab062e59-829e-474c-bb35-dab921ab2b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.733299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab062e59-829e-474c-bb35-dab921ab2b89 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.733925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab062e59-829e-474c-bb35-dab921ab2b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:23.735881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab062e59-829e-474c-bb35-dab921ab2b89 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:23.759880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1087/2152] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:23 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:23.764013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417d7bd-c5b9-476b-a8d3-0e84f0fb0132 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:23.769935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:23.780243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417d7bd-c5b9-476b-a8d3-0e84f0fb0132 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:23.792304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886d5f9-957c-4f61-8ede-92e6364934c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:23.792912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886d5f9-957c-4f61-8ede-92e6364934c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.793398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886d5f9-957c-4f61-8ede-92e6364934c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.793489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886d5f9-957c-4f61-8ede-92e6364934c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.794143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886d5f9-957c-4f61-8ede-92e6364934c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:23.796204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886d5f9-957c-4f61-8ede-92e6364934c0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:23.809201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1066/2153] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:23.812976 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab062e59-829e-474c-bb35-dab921ab2b89 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:23.823822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab062e59-829e-474c-bb35-dab921ab2b89 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:23.829447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab36895-6d1b-4d39-9f26-eac4b12ef03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:23.830379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab36895-6d1b-4d39-9f26-eac4b12ef03a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.830379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab36895-6d1b-4d39-9f26-eac4b12ef03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.830640 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab36895-6d1b-4d39-9f26-eac4b12ef03a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.831283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab36895-6d1b-4d39-9f26-eac4b12ef03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:23.832912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab36895-6d1b-4d39-9f26-eac4b12ef03a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:23.846409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1088/2154] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:23.850975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886d5f9-957c-4f61-8ede-92e6364934c0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:23.863410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886d5f9-957c-4f61-8ede-92e6364934c0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:23.869197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:23.870600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.871057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.871473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] PATH_INFO: `/v3/projects/2157af6d23784501945d61b94d818e89/users/3ea2a5bd7240428b812db2965c72ed5f/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.893264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2157af6d23784501945d61b94d818e89, user_id=3ea2a5bd7240428b812db2965c72ed5f, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:23.899044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:23.918454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1067/2155] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:23 2022] PUT /identity/v3/projects/2157af6d23784501945d61b94d818e89/users/3ea2a5bd7240428b812db2965c72ed5f/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:23.932167 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dd6a7-ab41-4d21-b25c-f173d02260aa None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:23.932471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dd6a7-ab41-4d21-b25c-f173d02260aa None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:23.932713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dd6a7-ab41-4d21-b25c-f173d02260aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:23.968548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-032dd6a7-ab41-4d21-b25c-f173d02260aa None None] MFA Rules not processed for user `3ea2a5bd7240428b812db2965c72ed5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:23.971893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-032dd6a7-ab41-4d21-b25c-f173d02260aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:24.074025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1089/2156] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:24.079604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:24.082419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:24.197477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477065f0-d981-49c3-9ead-2ba2410daf34 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:24.201144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591936a-46ec-4e2c-bb69-0593dfccf630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:24.202121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591936a-46ec-4e2c-bb69-0593dfccf630 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:24.202121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591936a-46ec-4e2c-bb69-0593dfccf630 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:24.202344 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591936a-46ec-4e2c-bb69-0593dfccf630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:24.211870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591936a-46ec-4e2c-bb69-0593dfccf630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:24.213339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591936a-46ec-4e2c-bb69-0593dfccf630 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:24.223602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1068/2157] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:26.400951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032dd6a7-ab41-4d21-b25c-f173d02260aa None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:26.402403 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-032dd6a7-ab41-4d21-b25c-f173d02260aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:26.465030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032dd6a7-ab41-4d21-b25c-f173d02260aa None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:26.469549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:26.470168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:26.470483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:26.470736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] PATH_INFO: `/v3/users/60ceb9f720f54a02a95fdbbc93d2396b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:26.471833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ceb9f720f54a02a95fdbbc93d2396b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:26.473802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:26.522986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ceb9f720f54a02a95fdbbc93d2396b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:26.530559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ceb9f720f54a02a95fdbbc93d2396b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:26.531295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ceb9f720f54a02a95fdbbc93d2396b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:26.538992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:26.545288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ceb9f720f54a02a95fdbbc93d2396b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:26.546147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ceb9f720f54a02a95fdbbc93d2396b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:26.560602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1090/2158] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:26 2022] DELETE /identity/v3/users/60ceb9f720f54a02a95fdbbc93d2396b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:26.565013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39691-fbba-4d0a-b51a-9b51416b70b2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:26.565568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39691-fbba-4d0a-b51a-9b51416b70b2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:26.565661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39691-fbba-4d0a-b51a-9b51416b70b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:26.584581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5be39691-fbba-4d0a-b51a-9b51416b70b2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:26.585889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5be39691-fbba-4d0a-b51a-9b51416b70b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:26.651535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1069/2159] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:26.658588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:26.659992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:26.715378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce144b5-cccd-41c6-aa72-37c144cc6e47 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:26.718988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ec20b-316a-4e6f-8025-ed0a77fba830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:26.719579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ec20b-316a-4e6f-8025-ed0a77fba830 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:26.719852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ec20b-316a-4e6f-8025-ed0a77fba830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:26.720122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ec20b-316a-4e6f-8025-ed0a77fba830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:26.728869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ec20b-316a-4e6f-8025-ed0a77fba830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:26.730326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ec20b-316a-4e6f-8025-ed0a77fba830 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:26.740253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1091/2160] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:27.008984 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be39691-fbba-4d0a-b51a-9b51416b70b2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:27.010431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5be39691-fbba-4d0a-b51a-9b51416b70b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:27.024762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be39691-fbba-4d0a-b51a-9b51416b70b2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:27.029501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:27.030139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:27.030415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:27.030730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] PATH_INFO: `/v3/projects/02952f4070d940febe8112379639d751` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:27.031804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] RBAC: Authorizing `identity:delete_project(project_id=02952f4070d940febe8112379639d751)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:27.033870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:27.086947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02952f4070d940febe8112379639d751', 'request_id': 'req-abfb3417-df01-4f02-b9df-de228270beae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.093368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02952f4070d940febe8112379639d751', 'request_id': 'req-abfb3417-df01-4f02-b9df-de228270beae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.106453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02952f4070d940febe8112379639d751', 'request_id': 'req-abfb3417-df01-4f02-b9df-de228270beae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.107912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:27.117484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] The token cache is being invalidate because project 02952f4070d940febe8112379639d751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:27.118183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.119009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.121174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1070/2161] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:27 2022] DELETE /identity/v3/projects/02952f4070d940febe8112379639d751 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:27.125276 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ec20b-316a-4e6f-8025-ed0a77fba830 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:27.135784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ec20b-316a-4e6f-8025-ed0a77fba830 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:27.194358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:27.195006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:27.195354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:27.195638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] PATH_INFO: `/v3/users/5489fc9126344d2b9aafe940331da31d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:27.196786 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5489fc9126344d2b9aafe940331da31d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:27.198920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:27.250993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5489fc9126344d2b9aafe940331da31d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.258945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5489fc9126344d2b9aafe940331da31d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.259552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5489fc9126344d2b9aafe940331da31d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.272125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5489fc9126344d2b9aafe940331da31d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.272822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-264b0b54-3d63-46e7-9d20-535d3ac4ee7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5489fc9126344d2b9aafe940331da31d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.284798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1092/2162] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:27 2022] DELETE /identity/v3/users/5489fc9126344d2b9aafe940331da31d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:27.556524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:27.558113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:27.573463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb3417-df01-4f02-b9df-de228270beae admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:27.630316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:27.631480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:27.631751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:27.632012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] PATH_INFO: `/v3/projects/9019dea15e0440dd82ee37ad30e1f816` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:27.633108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9019dea15e0440dd82ee37ad30e1f816)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:27.635107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:27.683620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9019dea15e0440dd82ee37ad30e1f816', 'request_id': 'req-c238a896-29af-44da-ab9b-816e9db7391d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.690285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9019dea15e0440dd82ee37ad30e1f816', 'request_id': 'req-c238a896-29af-44da-ab9b-816e9db7391d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.697065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9019dea15e0440dd82ee37ad30e1f816', 'request_id': 'req-c238a896-29af-44da-ab9b-816e9db7391d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.709467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] The token cache is being invalidate because project 9019dea15e0440dd82ee37ad30e1f816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:27.709749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.710358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c238a896-29af-44da-ab9b-816e9db7391d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:27.711823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1071/2163] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:27 2022] DELETE /identity/v3/projects/9019dea15e0440dd82ee37ad30e1f816 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:27.718934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66345e-142c-40c5-aab7-76d958aea6df None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:27.719445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66345e-142c-40c5-aab7-76d958aea6df None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:27.719538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66345e-142c-40c5-aab7-76d958aea6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:27.727342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:27.742875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1d66345e-142c-40c5-aab7-76d958aea6df None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:27.744081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1d66345e-142c-40c5-aab7-76d958aea6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:27.806038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1093/2164] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:27.814999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f93da-e6ce-4184-9001-abfb19baced1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:27.815287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f93da-e6ce-4184-9001-abfb19baced1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:27.815589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f93da-e6ce-4184-9001-abfb19baced1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:27.836286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c49f93da-e6ce-4184-9001-abfb19baced1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:27.837527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c49f93da-e6ce-4184-9001-abfb19baced1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:27.848611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1072/2165] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:27.849902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66345e-142c-40c5-aab7-76d958aea6df None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:27.859487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66345e-142c-40c5-aab7-76d958aea6df None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:27.866114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:27.866839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:27.867268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:27.867540 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:27.868559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:27.868911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:27.871190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:27.883151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1094/2166] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:27.887409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f93da-e6ce-4184-9001-abfb19baced1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:27.897174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f93da-e6ce-4184-9001-abfb19baced1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:27.901788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbb897-f24e-4af3-bfe3-baabc07d7056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:27.902441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbb897-f24e-4af3-bfe3-baabc07d7056 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:27.902716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbb897-f24e-4af3-bfe3-baabc07d7056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:27.902983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbb897-f24e-4af3-bfe3-baabc07d7056 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:27.903609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbb897-f24e-4af3-bfe3-baabc07d7056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:27.905524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbb897-f24e-4af3-bfe3-baabc07d7056 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:27.931706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1073/2167] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:27 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:27.935958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:27.946407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40781caf-c6d0-4488-a852-b669267ad920 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:27.999810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583ea06-b25a-4594-bf67-2481bedb96a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.000428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583ea06-b25a-4594-bf67-2481bedb96a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.000697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583ea06-b25a-4594-bf67-2481bedb96a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.000961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583ea06-b25a-4594-bf67-2481bedb96a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.001594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583ea06-b25a-4594-bf67-2481bedb96a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.003491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583ea06-b25a-4594-bf67-2481bedb96a8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.031203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1095/2168] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:27 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:28.032031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbb897-f24e-4af3-bfe3-baabc07d7056 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.044980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbb897-f24e-4af3-bfe3-baabc07d7056 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.049578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592820c-c1e3-4167-8bee-3075d2d78538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.050171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592820c-c1e3-4167-8bee-3075d2d78538 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.050447 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592820c-c1e3-4167-8bee-3075d2d78538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.050720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592820c-c1e3-4167-8bee-3075d2d78538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.051355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592820c-c1e3-4167-8bee-3075d2d78538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.053095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592820c-c1e3-4167-8bee-3075d2d78538 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.062780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1074/2169] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:28.066597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583ea06-b25a-4594-bf67-2481bedb96a8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.077353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583ea06-b25a-4594-bf67-2481bedb96a8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.081877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ccd44-60fa-4fdf-994a-6700ca5738b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.082468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ccd44-60fa-4fdf-994a-6700ca5738b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.082743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ccd44-60fa-4fdf-994a-6700ca5738b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.083013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ccd44-60fa-4fdf-994a-6700ca5738b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.084261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ccd44-60fa-4fdf-994a-6700ca5738b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.085878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ccd44-60fa-4fdf-994a-6700ca5738b1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.096418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1096/2170] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:28.100355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592820c-c1e3-4167-8bee-3075d2d78538 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.110694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592820c-c1e3-4167-8bee-3075d2d78538 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.115221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aba3b-2a2d-4279-b274-7aebd2e70022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.115867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aba3b-2a2d-4279-b274-7aebd2e70022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.116144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aba3b-2a2d-4279-b274-7aebd2e70022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.116414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aba3b-2a2d-4279-b274-7aebd2e70022 admin admin] PATH_INFO: `/v3/projects/d6350dc1339f4613827a4b3252b08652/users/0029ad652083401d8f87b4bea5d56fc6/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.131924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49aba3b-2a2d-4279-b274-7aebd2e70022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6350dc1339f4613827a4b3252b08652, user_id=0029ad652083401d8f87b4bea5d56fc6, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.134711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49aba3b-2a2d-4279-b274-7aebd2e70022 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.148230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1075/2171] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:28 2022] PUT /identity/v3/projects/d6350dc1339f4613827a4b3252b08652/users/0029ad652083401d8f87b4bea5d56fc6/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:28.158196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd70c-e2bf-4a66-9a18-e06a6b0c09b3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.158467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd70c-e2bf-4a66-9a18-e06a6b0c09b3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.158733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd70c-e2bf-4a66-9a18-e06a6b0c09b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.185388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3cbdd70c-e2bf-4a66-9a18-e06a6b0c09b3 None None] MFA Rules not processed for user `0029ad652083401d8f87b4bea5d56fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:28.186569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3cbdd70c-e2bf-4a66-9a18-e06a6b0c09b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:28.243878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:28.249362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1097/2172] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:28.254128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aba3b-2a2d-4279-b274-7aebd2e70022 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.276462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:28.320607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aba3b-2a2d-4279-b274-7aebd2e70022 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.325776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602724-abcd-40c0-93f1-76b4987ab0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.326462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602724-abcd-40c0-93f1-76b4987ab0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.326802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602724-abcd-40c0-93f1-76b4987ab0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.327136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d602724-abcd-40c0-93f1-76b4987ab0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.327825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d602724-abcd-40c0-93f1-76b4987ab0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.329810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d602724-abcd-40c0-93f1-76b4987ab0d9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.361203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1076/2173] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:28 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:28.362458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd70c-e2bf-4a66-9a18-e06a6b0c09b3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.373776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd70c-e2bf-4a66-9a18-e06a6b0c09b3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.431299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88839151-0189-40c0-b409-f0f3bc640d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.431906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88839151-0189-40c0-b409-f0f3bc640d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.432181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88839151-0189-40c0-b409-f0f3bc640d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.432456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88839151-0189-40c0-b409-f0f3bc640d72 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.433072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88839151-0189-40c0-b409-f0f3bc640d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.435032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88839151-0189-40c0-b409-f0f3bc640d72 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.455441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1098/2174] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:28 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:28.460008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602724-abcd-40c0-93f1-76b4987ab0d9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.470009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d602724-abcd-40c0-93f1-76b4987ab0d9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.475728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0e5e6-ddda-4160-8a44-5c8387e1218b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.476317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0e5e6-ddda-4160-8a44-5c8387e1218b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.476589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0e5e6-ddda-4160-8a44-5c8387e1218b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.476856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0e5e6-ddda-4160-8a44-5c8387e1218b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.477516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0e5e6-ddda-4160-8a44-5c8387e1218b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.479088 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0e5e6-ddda-4160-8a44-5c8387e1218b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.489550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1077/2175] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:28.494196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88839151-0189-40c0-b409-f0f3bc640d72 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.504130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88839151-0189-40c0-b409-f0f3bc640d72 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.509462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010015f9-17fa-48b3-b4c8-167409df8868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.510115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010015f9-17fa-48b3-b4c8-167409df8868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.510398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010015f9-17fa-48b3-b4c8-167409df8868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.510670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010015f9-17fa-48b3-b4c8-167409df8868 admin admin] PATH_INFO: `/v3/projects/9758a78e293e4f01828c0f0b1c56d293/users/ef20074980794942b27381305ae82bfb/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.525669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010015f9-17fa-48b3-b4c8-167409df8868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9758a78e293e4f01828c0f0b1c56d293, user_id=ef20074980794942b27381305ae82bfb, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.528435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010015f9-17fa-48b3-b4c8-167409df8868 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.541889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1099/2176] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:28 2022] PUT /identity/v3/projects/9758a78e293e4f01828c0f0b1c56d293/users/ef20074980794942b27381305ae82bfb/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:28.545801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0e5e6-ddda-4160-8a44-5c8387e1218b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.555634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0e5e6-ddda-4160-8a44-5c8387e1218b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.611350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837957c-6def-4732-91af-d6d944704c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.611981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837957c-6def-4732-91af-d6d944704c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.612202 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837957c-6def-4732-91af-d6d944704c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.612465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837957c-6def-4732-91af-d6d944704c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.613391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837957c-6def-4732-91af-d6d944704c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.615550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837957c-6def-4732-91af-d6d944704c45 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.625011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1078/2177] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:28.629180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010015f9-17fa-48b3-b4c8-167409df8868 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.639377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010015f9-17fa-48b3-b4c8-167409df8868 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.643935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.644566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.644841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.645109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] PATH_INFO: `/v3/projects/9758a78e293e4f01828c0f0b1c56d293/users/ef20074980794942b27381305ae82bfb/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.647014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9758a78e293e4f01828c0f0b1c56d293, user_id=ef20074980794942b27381305ae82bfb, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.649663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.659500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1100/2178] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:28 2022] PUT /identity/v3/projects/9758a78e293e4f01828c0f0b1c56d293/users/ef20074980794942b27381305ae82bfb/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:28.670047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb5c74-3ef1-4390-9975-77a21fd84a83 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.670479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb5c74-3ef1-4390-9975-77a21fd84a83 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.670664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb5c74-3ef1-4390-9975-77a21fd84a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.698915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5bbb5c74-3ef1-4390-9975-77a21fd84a83 None None] MFA Rules not processed for user `ef20074980794942b27381305ae82bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:28.700177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5bbb5c74-3ef1-4390-9975-77a21fd84a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:28.766496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1079/2179] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:28.776943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:28.778656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:28.835656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a77a0-537e-4d49-80d5-6a2f9cf3b128 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:28.839337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b7af2-4565-4345-9db1-54f5e8867874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:28.839928 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7af2-4565-4345-9db1-54f5e8867874 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:28.840203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7af2-4565-4345-9db1-54f5e8867874 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:28.840471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7af2-4565-4345-9db1-54f5e8867874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:28.899747 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b7af2-4565-4345-9db1-54f5e8867874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:28.901185 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b7af2-4565-4345-9db1-54f5e8867874 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:28.910200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1101/2180] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:30.394634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb5c74-3ef1-4390-9975-77a21fd84a83 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:30.396933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5bbb5c74-3ef1-4390-9975-77a21fd84a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:30.418594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:30.480723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb5c74-3ef1-4390-9975-77a21fd84a83 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:30.490250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:30.490970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:30.491318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:30.491694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] PATH_INFO: `/v3/users/ae477ee526df42af84bd782747722205` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:30.493040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae477ee526df42af84bd782747722205)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:30.495240 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:30.560519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae477ee526df42af84bd782747722205'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:30.570018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae477ee526df42af84bd782747722205'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:30.570607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae477ee526df42af84bd782747722205'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:30.581980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae477ee526df42af84bd782747722205'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:30.598158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae477ee526df42af84bd782747722205'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:30.606719 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1080/2181] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:30 2022] DELETE /identity/v3/users/ae477ee526df42af84bd782747722205 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:30.612178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bcf40-40d9-431a-a647-f57cd8289aa8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:30.613550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bcf40-40d9-431a-a647-f57cd8289aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:30.614014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bcf40-40d9-431a-a647-f57cd8289aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:30.626892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:30.649963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7e6bcf40-40d9-431a-a647-f57cd8289aa8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:30.651660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7e6bcf40-40d9-431a-a647-f57cd8289aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:30.744805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1102/2182] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:30.758488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:30.759924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:30.835139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f54d5-65d4-4750-94e6-c742a3b0afe7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:30.838811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:30.839508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:30.839787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:30.840057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:30.849752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:30.851212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:30.860776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1081/2183] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:31.127193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bcf40-40d9-431a-a647-f57cd8289aa8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:31.128821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7e6bcf40-40d9-431a-a647-f57cd8289aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:31.143183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bcf40-40d9-431a-a647-f57cd8289aa8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:31.148699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:31.149390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:31.149700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:31.149921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] PATH_INFO: `/v3/projects/9036bd761a1e42c285cf20965136efe9` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:31.151142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9036bd761a1e42c285cf20965136efe9)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:31.153362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:31.200052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9036bd761a1e42c285cf20965136efe9', 'request_id': 'req-0d73d424-29a5-44ca-bbad-ee9f401534b4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:31.205599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9036bd761a1e42c285cf20965136efe9', 'request_id': 'req-0d73d424-29a5-44ca-bbad-ee9f401534b4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:31.211866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9036bd761a1e42c285cf20965136efe9', 'request_id': 'req-0d73d424-29a5-44ca-bbad-ee9f401534b4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:31.225541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] The token cache is being invalidate because project 9036bd761a1e42c285cf20965136efe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:31.225907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:31.226513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0d73d424-29a5-44ca-bbad-ee9f401534b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:31.228014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1103/2184] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:31 2022] DELETE /identity/v3/projects/9036bd761a1e42c285cf20965136efe9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:32.980497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:32.981912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:32.995114 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27439cdb-51bd-4803-859a-317b38e4a396 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:33.027500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:33.051498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:33.052838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.053148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.053455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] PATH_INFO: `/v3/users/3ea2a5bd7240428b812db2965c72ed5f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.054565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea2a5bd7240428b812db2965c72ed5f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:33.056460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:33.105577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea2a5bd7240428b812db2965c72ed5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.114473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea2a5bd7240428b812db2965c72ed5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.115715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea2a5bd7240428b812db2965c72ed5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.123368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea2a5bd7240428b812db2965c72ed5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.136670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea2a5bd7240428b812db2965c72ed5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.142982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1082/2185] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:32 2022] DELETE /identity/v3/users/3ea2a5bd7240428b812db2965c72ed5f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:33.147730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaa83b-b6e1-44fd-a280-7bcce97664a8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.148186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaa83b-b6e1-44fd-a280-7bcce97664a8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.148264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaa83b-b6e1-44fd-a280-7bcce97664a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.168762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-eefaa83b-b6e1-44fd-a280-7bcce97664a8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:33.170070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-eefaa83b-b6e1-44fd-a280-7bcce97664a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:33.226051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:33.235325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:33.238431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1104/2186] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:33.248915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaa83b-b6e1-44fd-a280-7bcce97664a8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:33.250852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-eefaa83b-b6e1-44fd-a280-7bcce97664a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:33.283520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:33.301080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd75f77-1013-4855-969c-7d8582f61375 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:33.304730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecae2f-2597-460c-8f80-b9e2e3f1bb54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:33.305369 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecae2f-2597-460c-8f80-b9e2e3f1bb54 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.305647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecae2f-2597-460c-8f80-b9e2e3f1bb54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.305915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecae2f-2597-460c-8f80-b9e2e3f1bb54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.307643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-96ecae2f-2597-460c-8f80-b9e2e3f1bb54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:33.309663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaa83b-b6e1-44fd-a280-7bcce97664a8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:33.313247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:33.313843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.314115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.314381 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.323695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:33.325115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:33.335007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1105/2187] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:39:33.374263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecae2f-2597-460c-8f80-b9e2e3f1bb54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:33.375701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecae2f-2597-460c-8f80-b9e2e3f1bb54 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:33.385921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1083/2188] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:33.617556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:33.619714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:33.637661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056bbb4-9688-4d5c-aa7c-ec5e1b68acd5 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:33.647154 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:33.647678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.647950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.648217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] PATH_INFO: `/v3/projects/2157af6d23784501945d61b94d818e89` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.649849 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2157af6d23784501945d61b94d818e89)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:33.651708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:33.727707 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2157af6d23784501945d61b94d818e89', 'request_id': 'req-5461580d-4205-4e28-af10-1c2bed391521'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.734269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2157af6d23784501945d61b94d818e89', 'request_id': 'req-5461580d-4205-4e28-af10-1c2bed391521'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.741740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2157af6d23784501945d61b94d818e89', 'request_id': 'req-5461580d-4205-4e28-af10-1c2bed391521'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.759182 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] The token cache is being invalidate because project 2157af6d23784501945d61b94d818e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:33.759484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.760102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5461580d-4205-4e28-af10-1c2bed391521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:33.765399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1106/2189] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:33 2022] DELETE /identity/v3/projects/2157af6d23784501945d61b94d818e89 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:33.766927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c03fb-d61c-41cb-bc88-5e5305bd9fa0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.767449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c03fb-d61c-41cb-bc88-5e5305bd9fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.767923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c03fb-d61c-41cb-bc88-5e5305bd9fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.786904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:33.796034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6e0c03fb-d61c-41cb-bc88-5e5305bd9fa0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:33.798188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6e0c03fb-d61c-41cb-bc88-5e5305bd9fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:33.860826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1084/2190] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:33.869980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef1b78-c8b7-41e7-b2f2-28b9e3907198 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.870269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef1b78-c8b7-41e7-b2f2-28b9e3907198 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.870539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef1b78-c8b7-41e7-b2f2-28b9e3907198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.889645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-24ef1b78-c8b7-41e7-b2f2-28b9e3907198 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:33.890833 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-24ef1b78-c8b7-41e7-b2f2-28b9e3907198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:33.898549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1107/2191] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:33.902731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c03fb-d61c-41cb-bc88-5e5305bd9fa0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:33.912807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c03fb-d61c-41cb-bc88-5e5305bd9fa0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:33.917264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:33.917854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.918129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.918396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.919118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:33.919431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:33.921006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:33.932857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1085/2192] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:33.936979 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef1b78-c8b7-41e7-b2f2-28b9e3907198 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:33.946078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef1b78-c8b7-41e7-b2f2-28b9e3907198 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:33.950665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e19b6-3bd3-4be3-8ff9-5189516a60c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:33.951239 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e19b6-3bd3-4be3-8ff9-5189516a60c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:33.951511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e19b6-3bd3-4be3-8ff9-5189516a60c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:33.951774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e19b6-3bd3-4be3-8ff9-5189516a60c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:33.952416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e19b6-3bd3-4be3-8ff9-5189516a60c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:33.954327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e19b6-3bd3-4be3-8ff9-5189516a60c9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:33.981908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1108/2193] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:33 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:33.985773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:33.996844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfe7e1-5102-40d2-b536-4651c767c19d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.056258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736ffd-3516-44c0-8af2-fdbdff292dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.056865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736ffd-3516-44c0-8af2-fdbdff292dea admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.057150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736ffd-3516-44c0-8af2-fdbdff292dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.057448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736ffd-3516-44c0-8af2-fdbdff292dea admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.058076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa736ffd-3516-44c0-8af2-fdbdff292dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.059990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa736ffd-3516-44c0-8af2-fdbdff292dea admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.080491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1086/2194] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:33 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:34.085203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e19b6-3bd3-4be3-8ff9-5189516a60c9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.095815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e19b6-3bd3-4be3-8ff9-5189516a60c9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.100750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbafde-297c-4d21-83a3-563e1b256288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.101384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbafde-297c-4d21-83a3-563e1b256288 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.101665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbafde-297c-4d21-83a3-563e1b256288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.102757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbafde-297c-4d21-83a3-563e1b256288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.103701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbafde-297c-4d21-83a3-563e1b256288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.105593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbafde-297c-4d21-83a3-563e1b256288 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.134712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1109/2195] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:39:34.138747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736ffd-3516-44c0-8af2-fdbdff292dea admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.149327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736ffd-3516-44c0-8af2-fdbdff292dea admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.153902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fe729-4d32-4240-9116-fbf2bdbbd422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.154493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe729-4d32-4240-9116-fbf2bdbbd422 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.154772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe729-4d32-4240-9116-fbf2bdbbd422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.155039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe729-4d32-4240-9116-fbf2bdbbd422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.155696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24fe729-4d32-4240-9116-fbf2bdbbd422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.157290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24fe729-4d32-4240-9116-fbf2bdbbd422 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.167696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1087/2196] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:34.171954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbafde-297c-4d21-83a3-563e1b256288 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.183160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbafde-297c-4d21-83a3-563e1b256288 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.187828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e3a6d-8e74-4a70-9aa2-401588d13c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.188491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e3a6d-8e74-4a70-9aa2-401588d13c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.188749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e3a6d-8e74-4a70-9aa2-401588d13c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.189019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e3a6d-8e74-4a70-9aa2-401588d13c98 admin admin] PATH_INFO: `/v3/projects/434dc77082bd461199e9ec803bb84716/users/8d7d5e0f6ca341179d70c6ef426da35e/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.204745 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e3a6d-8e74-4a70-9aa2-401588d13c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=434dc77082bd461199e9ec803bb84716, user_id=8d7d5e0f6ca341179d70c6ef426da35e, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.207501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e3a6d-8e74-4a70-9aa2-401588d13c98 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.220822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1110/2197] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:34 2022] PUT /identity/v3/projects/434dc77082bd461199e9ec803bb84716/users/8d7d5e0f6ca341179d70c6ef426da35e/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:34.231501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534c4ff-6dba-4c1d-9de9-fdd41aa7fccf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.231789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534c4ff-6dba-4c1d-9de9-fdd41aa7fccf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.232110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534c4ff-6dba-4c1d-9de9-fdd41aa7fccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.269640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e534c4ff-6dba-4c1d-9de9-fdd41aa7fccf None None] MFA Rules not processed for user `8d7d5e0f6ca341179d70c6ef426da35e`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:34.276328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e534c4ff-6dba-4c1d-9de9-fdd41aa7fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:34.369310 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1088/2198] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:34.370439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e3a6d-8e74-4a70-9aa2-401588d13c98 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.428243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:34.440873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e3a6d-8e74-4a70-9aa2-401588d13c98 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.446355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960240b-4b19-4c0e-b779-751371f6e0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.446955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960240b-4b19-4c0e-b779-751371f6e0db admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.447229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960240b-4b19-4c0e-b779-751371f6e0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.447497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960240b-4b19-4c0e-b779-751371f6e0db admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.448105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960240b-4b19-4c0e-b779-751371f6e0db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.450031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960240b-4b19-4c0e-b779-751371f6e0db admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.475327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1111/2199] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:34 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:34.479349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534c4ff-6dba-4c1d-9de9-fdd41aa7fccf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.484869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:34.492475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534c4ff-6dba-4c1d-9de9-fdd41aa7fccf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.548143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15712-7da4-49f0-aac0-ab1cba966d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.548750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15712-7da4-49f0-aac0-ab1cba966d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.549022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15712-7da4-49f0-aac0-ab1cba966d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.549314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15712-7da4-49f0-aac0-ab1cba966d42 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.549935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15712-7da4-49f0-aac0-ab1cba966d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.551821 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15712-7da4-49f0-aac0-ab1cba966d42 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.572372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1089/2200] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:34 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:34.576455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960240b-4b19-4c0e-b779-751371f6e0db admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.587148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960240b-4b19-4c0e-b779-751371f6e0db admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.591807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f69a7-d861-4561-927a-8878a4cbc7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.592392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f69a7-d861-4561-927a-8878a4cbc7de admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.592666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f69a7-d861-4561-927a-8878a4cbc7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.592932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f69a7-d861-4561-927a-8878a4cbc7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.593593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f69a7-d861-4561-927a-8878a4cbc7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.595170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f69a7-d861-4561-927a-8878a4cbc7de admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.605425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1112/2201] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:34.609569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15712-7da4-49f0-aac0-ab1cba966d42 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.619690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15712-7da4-49f0-aac0-ab1cba966d42 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.624288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9e05c-1d1f-40cb-8df9-8b23a98ed7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.624876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9e05c-1d1f-40cb-8df9-8b23a98ed7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.625154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9e05c-1d1f-40cb-8df9-8b23a98ed7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.625463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9e05c-1d1f-40cb-8df9-8b23a98ed7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.626102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9e05c-1d1f-40cb-8df9-8b23a98ed7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.627744 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9e05c-1d1f-40cb-8df9-8b23a98ed7ab admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.637968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1090/2202] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:34.641864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f69a7-d861-4561-927a-8878a4cbc7de admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.653716 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f69a7-d861-4561-927a-8878a4cbc7de admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.658337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19817290-3a34-433a-8b78-25a291324557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.658978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817290-3a34-433a-8b78-25a291324557 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.659255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817290-3a34-433a-8b78-25a291324557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.659532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817290-3a34-433a-8b78-25a291324557 admin admin] PATH_INFO: `/v3/projects/f59577ab879946c7954e6624de4eb6f0/users/f4578ca2593142e8badc04cfd3faf70b/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.674350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19817290-3a34-433a-8b78-25a291324557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59577ab879946c7954e6624de4eb6f0, user_id=f4578ca2593142e8badc04cfd3faf70b, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.677562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19817290-3a34-433a-8b78-25a291324557 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:34.693845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1113/2203] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:34 2022] PUT /identity/v3/projects/f59577ab879946c7954e6624de4eb6f0/users/f4578ca2593142e8badc04cfd3faf70b/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:34.704891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62e0e-dfe0-4a20-9011-8112a45cd577 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.705408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62e0e-dfe0-4a20-9011-8112a45cd577 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.705678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62e0e-dfe0-4a20-9011-8112a45cd577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.736992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2ce62e0e-dfe0-4a20-9011-8112a45cd577 None None] MFA Rules not processed for user `f4578ca2593142e8badc04cfd3faf70b`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:34.738442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2ce62e0e-dfe0-4a20-9011-8112a45cd577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:34.833502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1091/2204] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:34.836287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19817290-3a34-433a-8b78-25a291324557 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:34.955105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19817290-3a34-433a-8b78-25a291324557 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:34.963209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cce4d8-561d-4fd1-98e5-9a4c9db8b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:34.963980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cce4d8-561d-4fd1-98e5-9a4c9db8b527 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:34.964103 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cce4d8-561d-4fd1-98e5-9a4c9db8b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:34.964373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cce4d8-561d-4fd1-98e5-9a4c9db8b527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:34.964959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cce4d8-561d-4fd1-98e5-9a4c9db8b527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:34.967034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cce4d8-561d-4fd1-98e5-9a4c9db8b527 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:35.002360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1114/2205] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:34 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:35.003368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce62e0e-dfe0-4a20-9011-8112a45cd577 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:35.019213 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce62e0e-dfe0-4a20-9011-8112a45cd577 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:35.079613 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34288b28-8b17-443a-a021-28787846af43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:35.080219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288b28-8b17-443a-a021-28787846af43 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:35.080491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288b28-8b17-443a-a021-28787846af43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:35.080756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288b28-8b17-443a-a021-28787846af43 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:35.081394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34288b28-8b17-443a-a021-28787846af43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:35.083340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34288b28-8b17-443a-a021-28787846af43 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:35.100797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:35.106475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1092/2206] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:35 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:35.110488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cce4d8-561d-4fd1-98e5-9a4c9db8b527 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:35.122246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cce4d8-561d-4fd1-98e5-9a4c9db8b527 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:35.127017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2ad84-5d31-4733-abf0-e70e34ff2925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:35.127609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2ad84-5d31-4733-abf0-e70e34ff2925 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:35.127884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2ad84-5d31-4733-abf0-e70e34ff2925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:35.128148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2ad84-5d31-4733-abf0-e70e34ff2925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:35.128778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2ad84-5d31-4733-abf0-e70e34ff2925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:35.131231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2ad84-5d31-4733-abf0-e70e34ff2925 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:35.141858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1115/2207] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:35.146259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34288b28-8b17-443a-a021-28787846af43 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:35.158995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34288b28-8b17-443a-a021-28787846af43 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:35.164137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bde4c-539b-488c-8d7b-7692244401ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:35.164795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bde4c-539b-488c-8d7b-7692244401ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:35.165067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bde4c-539b-488c-8d7b-7692244401ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:35.165366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bde4c-539b-488c-8d7b-7692244401ea admin admin] PATH_INFO: `/v3/projects/4c22cddd2942410985d2a8878e022af4/users/89e0070c67b54fc5b40b80528bd8f533/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:35.181345 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bde4c-539b-488c-8d7b-7692244401ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c22cddd2942410985d2a8878e022af4, user_id=89e0070c67b54fc5b40b80528bd8f533, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:35.184137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bde4c-539b-488c-8d7b-7692244401ea admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:35.198093 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1093/2208] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:35 2022] PUT /identity/v3/projects/4c22cddd2942410985d2a8878e022af4/users/89e0070c67b54fc5b40b80528bd8f533/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:35.202550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2ad84-5d31-4733-abf0-e70e34ff2925 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:35.213694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2ad84-5d31-4733-abf0-e70e34ff2925 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:35.273547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbb92f-99d7-4dc9-bb11-228822f98f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:35.274059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb92f-99d7-4dc9-bb11-228822f98f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:35.274335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb92f-99d7-4dc9-bb11-228822f98f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:35.274629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb92f-99d7-4dc9-bb11-228822f98f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:35.275266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbb92f-99d7-4dc9-bb11-228822f98f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:35.276869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbb92f-99d7-4dc9-bb11-228822f98f61 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:35.287295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1116/2209] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:35.291817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bde4c-539b-488c-8d7b-7692244401ea admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:35.302585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bde4c-539b-488c-8d7b-7692244401ea admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:35.307500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:35.308144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:35.308418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:35.308691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] PATH_INFO: `/v3/projects/4c22cddd2942410985d2a8878e022af4/users/89e0070c67b54fc5b40b80528bd8f533/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:35.310606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c22cddd2942410985d2a8878e022af4, user_id=89e0070c67b54fc5b40b80528bd8f533, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:35.313260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:35.324200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1094/2210] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:35 2022] PUT /identity/v3/projects/4c22cddd2942410985d2a8878e022af4/users/89e0070c67b54fc5b40b80528bd8f533/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:35.335047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9b560-429e-4ec8-929a-f99c3db16748 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:35.335404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9b560-429e-4ec8-929a-f99c3db16748 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:35.335679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9b560-429e-4ec8-929a-f99c3db16748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:35.356148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:35.365240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-46d9b560-429e-4ec8-929a-f99c3db16748 None None] MFA Rules not processed for user `89e0070c67b54fc5b40b80528bd8f533`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:35.366618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-46d9b560-429e-4ec8-929a-f99c3db16748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:35.434085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1117/2211] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:35.446885 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:35.448562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:35.508685 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f857475-a785-4efe-9827-12b7349167cd admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:35.512247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efd943-312b-47a7-aa38-b9e1fafade88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:35.512847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efd943-312b-47a7-aa38-b9e1fafade88 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:35.513120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efd943-312b-47a7-aa38-b9e1fafade88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:35.513416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efd943-312b-47a7-aa38-b9e1fafade88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:35.571978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efd943-312b-47a7-aa38-b9e1fafade88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:35.573513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efd943-312b-47a7-aa38-b9e1fafade88 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:35.583084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1095/2212] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:37.241117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9b560-429e-4ec8-929a-f99c3db16748 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:37.242537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-46d9b560-429e-4ec8-929a-f99c3db16748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:37.260881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9b560-429e-4ec8-929a-f99c3db16748 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:37.264946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:37.265817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:37.266192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:37.266552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:37.344132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:37.345893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:37.357471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1118/2213] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:37.953492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efd943-312b-47a7-aa38-b9e1fafade88 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:37.962285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efd943-312b-47a7-aa38-b9e1fafade88 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:37.966235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3936fa-114c-47c3-b638-fded406c1f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:37.966860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3936fa-114c-47c3-b638-fded406c1f07 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:37.967137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3936fa-114c-47c3-b638-fded406c1f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:37.967428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3936fa-114c-47c3-b638-fded406c1f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:37.976619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3936fa-114c-47c3-b638-fded406c1f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:37.978127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3936fa-114c-47c3-b638-fded406c1f07 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:37.987875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1096/2214] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:38.387151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:38.388578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:38.456545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3292d-cfca-4ef6-bd33-1f9c0a98b37b service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:38.462673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:38.463380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:38.463730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:38.464071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] PATH_INFO: `/v3/users/0029ad652083401d8f87b4bea5d56fc6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:38.465293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0029ad652083401d8f87b4bea5d56fc6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:38.467400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:38.531886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:38.563056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0029ad652083401d8f87b4bea5d56fc6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:38.580027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0029ad652083401d8f87b4bea5d56fc6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:38.580800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0029ad652083401d8f87b4bea5d56fc6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:38.593474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0029ad652083401d8f87b4bea5d56fc6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:38.594090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0029ad652083401d8f87b4bea5d56fc6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:38.606934 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1119/2215] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:38 2022] DELETE /identity/v3/users/0029ad652083401d8f87b4bea5d56fc6 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:38.611370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c30f4-608c-4417-a0ec-2c114f7b7968 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:38.611651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c30f4-608c-4417-a0ec-2c114f7b7968 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:38.611927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c30f4-608c-4417-a0ec-2c114f7b7968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:38.631226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a70c30f4-608c-4417-a0ec-2c114f7b7968 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:38.632407 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a70c30f4-608c-4417-a0ec-2c114f7b7968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:38.694340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1097/2216] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:38.704267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:38.705812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:38.763686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51655ae-c1a9-4918-9797-cc5af896820f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:38.768198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8ebc7-97c4-4608-89df-6c63d159c1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:38.768803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8ebc7-97c4-4608-89df-6c63d159c1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:38.769118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8ebc7-97c4-4608-89df-6c63d159c1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:38.769422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8ebc7-97c4-4608-89df-6c63d159c1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:38.780073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8ebc7-97c4-4608-89df-6c63d159c1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:38.781574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8ebc7-97c4-4608-89df-6c63d159c1fd service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:38.792494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1120/2217] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:39.065607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c30f4-608c-4417-a0ec-2c114f7b7968 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:39.067130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a70c30f4-608c-4417-a0ec-2c114f7b7968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:39.075532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:39.079857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c30f4-608c-4417-a0ec-2c114f7b7968 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:39.085216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:39.085673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:39.085965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:39.086787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] PATH_INFO: `/v3/projects/d6350dc1339f4613827a4b3252b08652` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:39.088441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6350dc1339f4613827a4b3252b08652)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:39.090851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:39.150728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6350dc1339f4613827a4b3252b08652', 'request_id': 'req-6ab9cf8e-e050-4d44-8a10-918719a89e8d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.156891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6350dc1339f4613827a4b3252b08652', 'request_id': 'req-6ab9cf8e-e050-4d44-8a10-918719a89e8d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.164264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6350dc1339f4613827a4b3252b08652', 'request_id': 'req-6ab9cf8e-e050-4d44-8a10-918719a89e8d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.178287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] The token cache is being invalidate because project d6350dc1339f4613827a4b3252b08652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:39.178571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.179190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.180690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1098/2218] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:39 2022] DELETE /identity/v3/projects/d6350dc1339f4613827a4b3252b08652 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:39.184773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8ebc7-97c4-4608-89df-6c63d159c1fd service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:39.195162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8ebc7-97c4-4608-89df-6c63d159c1fd service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:39.256565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:39.257198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:39.257506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:39.257776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] PATH_INFO: `/v3/users/ef20074980794942b27381305ae82bfb` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:39.258918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef20074980794942b27381305ae82bfb)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:39.260862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:39.310096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef20074980794942b27381305ae82bfb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.318100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef20074980794942b27381305ae82bfb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.319106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef20074980794942b27381305ae82bfb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.324113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:39.334950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef20074980794942b27381305ae82bfb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.335949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb765ae0-a2b4-4f41-9baa-74c44723001a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef20074980794942b27381305ae82bfb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.356933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1121/2219] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:39 2022] DELETE /identity/v3/users/ef20074980794942b27381305ae82bfb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:39.641754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:39.643538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:39.659661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9cf8e-e050-4d44-8a10-918719a89e8d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:39.738212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:39.738824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:39.739096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:39.739917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] PATH_INFO: `/v3/projects/9758a78e293e4f01828c0f0b1c56d293` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:39.741079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9758a78e293e4f01828c0f0b1c56d293)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:39.743273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:39.809896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9758a78e293e4f01828c0f0b1c56d293', 'request_id': 'req-09d33b33-6606-4ee9-9294-1cd0215b12e0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.816729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9758a78e293e4f01828c0f0b1c56d293', 'request_id': 'req-09d33b33-6606-4ee9-9294-1cd0215b12e0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.823299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9758a78e293e4f01828c0f0b1c56d293', 'request_id': 'req-09d33b33-6606-4ee9-9294-1cd0215b12e0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.839449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] The token cache is being invalidate because project 9758a78e293e4f01828c0f0b1c56d293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:39.839745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.840388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:39.842203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1099/2220] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:39 2022] DELETE /identity/v3/projects/9758a78e293e4f01828c0f0b1c56d293 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:39.849845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b055d9d-8995-4965-8358-a694c231b993 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:39.850282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b055d9d-8995-4965-8358-a694c231b993 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:39.850381 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b055d9d-8995-4965-8358-a694c231b993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:39.910388 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6b055d9d-8995-4965-8358-a694c231b993 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:39.911683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6b055d9d-8995-4965-8358-a694c231b993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:40.008296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.009889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:40.018818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1122/2221] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:40.027133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88052f-fc56-450e-b2fd-1abeb57518b8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.028340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88052f-fc56-450e-b2fd-1abeb57518b8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.028684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88052f-fc56-450e-b2fd-1abeb57518b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.044373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:40.089133 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9b88052f-fc56-450e-b2fd-1abeb57518b8 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:40.090586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9b88052f-fc56-450e-b2fd-1abeb57518b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:40.093453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d33b33-6606-4ee9-9294-1cd0215b12e0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.105036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1123/2222] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:40.108499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88052f-fc56-450e-b2fd-1abeb57518b8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.111098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.111976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.112221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.112754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.116755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:40.120651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88052f-fc56-450e-b2fd-1abeb57518b8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.127735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.128345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.128618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.128880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.129573 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:40.129880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.131528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.146018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1124/2223] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:40.150485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.169899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfc85f-dfc2-4a36-9f57-32379817d348 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.178577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e908e5-1b59-4f1f-859c-8eb9310e07bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.179534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e908e5-1b59-4f1f-859c-8eb9310e07bf admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.180401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e908e5-1b59-4f1f-859c-8eb9310e07bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.180744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e908e5-1b59-4f1f-859c-8eb9310e07bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.181808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e908e5-1b59-4f1f-859c-8eb9310e07bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.184845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e908e5-1b59-4f1f-859c-8eb9310e07bf admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.203269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.205029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.217212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1100/2224] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:39:40.232352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1125/2225] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:40 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:40.234486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.249423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22c71c-5d48-432e-9eb7-80b1d35b7e41 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.325505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f2d3-697e-48e5-9729-abd6f1df5a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.326370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f2d3-697e-48e5-9729-abd6f1df5a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.326718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f2d3-697e-48e5-9729-abd6f1df5a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.327060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f2d3-697e-48e5-9729-abd6f1df5a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.327805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f2d3-697e-48e5-9729-abd6f1df5a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.329983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f2d3-697e-48e5-9729-abd6f1df5a49 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.360442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1101/2226] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:40 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:40.366060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e908e5-1b59-4f1f-859c-8eb9310e07bf admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.376933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e908e5-1b59-4f1f-859c-8eb9310e07bf admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.382512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029762c0-b815-45b8-93cb-2e9fd1802643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.383229 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029762c0-b815-45b8-93cb-2e9fd1802643 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.383501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029762c0-b815-45b8-93cb-2e9fd1802643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.383763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029762c0-b815-45b8-93cb-2e9fd1802643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.384414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029762c0-b815-45b8-93cb-2e9fd1802643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.386072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029762c0-b815-45b8-93cb-2e9fd1802643 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.398002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1126/2227] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:40.401946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f2d3-697e-48e5-9729-abd6f1df5a49 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.413671 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f2d3-697e-48e5-9729-abd6f1df5a49 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.418657 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c0826-ed36-420c-bbbd-871e88ab6b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.419326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c0826-ed36-420c-bbbd-871e88ab6b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.419603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c0826-ed36-420c-bbbd-871e88ab6b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.419871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c0826-ed36-420c-bbbd-871e88ab6b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.420518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c0826-ed36-420c-bbbd-871e88ab6b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.422160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c0826-ed36-420c-bbbd-871e88ab6b99 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.432915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1102/2228] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:40.437217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029762c0-b815-45b8-93cb-2e9fd1802643 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.449062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029762c0-b815-45b8-93cb-2e9fd1802643 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.453706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.454360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.454639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.454926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] PATH_INFO: `/v3/projects/867bb71226f94a928575748de9363fa0/users/a92cd5f5081f46b1b430ca575748fc81/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.481826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=867bb71226f94a928575748de9363fa0, user_id=a92cd5f5081f46b1b430ca575748fc81, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.484476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.511870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1127/2229] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:40 2022] PUT /identity/v3/projects/867bb71226f94a928575748de9363fa0/users/a92cd5f5081f46b1b430ca575748fc81/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:40.524420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b941f-6235-4e45-baed-991709376247 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.524944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b941f-6235-4e45-baed-991709376247 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.526083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b941f-6235-4e45-baed-991709376247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.584569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-370b941f-6235-4e45-baed-991709376247 None None] MFA Rules not processed for user `a92cd5f5081f46b1b430ca575748fc81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:40.585844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-370b941f-6235-4e45-baed-991709376247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:40.639806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:40.689993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1103/2230] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:40.699288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.700506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:40.764513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19040a40-646f-44f9-a762-f7872ebdeb38 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.768695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fd544-8c08-40c5-9bb0-c637de3c8aea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.769692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd544-8c08-40c5-9bb0-c637de3c8aea service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.769692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd544-8c08-40c5-9bb0-c637de3c8aea service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.769940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd544-8c08-40c5-9bb0-c637de3c8aea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.779318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fd544-8c08-40c5-9bb0-c637de3c8aea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.782735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fd544-8c08-40c5-9bb0-c637de3c8aea service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.793478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1128/2231] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:40.819862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b941f-6235-4e45-baed-991709376247 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:40.897917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b941f-6235-4e45-baed-991709376247 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:40.903873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:40.904131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:40.904463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:40.904790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] PATH_INFO: `/v3/users/a92cd5f5081f46b1b430ca575748fc81` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:40.906559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a92cd5f5081f46b1b430ca575748fc81)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:40.909004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:40.972952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a92cd5f5081f46b1b430ca575748fc81'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:40.983058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a92cd5f5081f46b1b430ca575748fc81'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:40.983753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a92cd5f5081f46b1b430ca575748fc81'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:40.993837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a92cd5f5081f46b1b430ca575748fc81'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:41.019877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a92cd5f5081f46b1b430ca575748fc81'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:41.029966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1104/2232] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:40 2022] DELETE /identity/v3/users/a92cd5f5081f46b1b430ca575748fc81 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:41.034550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b5b5-d219-4c4d-a0a6-3e02755b7097 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:41.035077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b5b5-d219-4c4d-a0a6-3e02755b7097 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:41.035785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b5b5-d219-4c4d-a0a6-3e02755b7097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:41.043746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:41.059285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7b90b5b5-d219-4c4d-a0a6-3e02755b7097 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:41.061056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7b90b5b5-d219-4c4d-a0a6-3e02755b7097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:41.143385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1129/2233] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:41.154967 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:41.157162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:41.223191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47331637-c3a2-4d00-a3e0-aea4653600e8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:41.226950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82c2e6-a798-420c-be5a-e6c6e230ebb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:41.227565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82c2e6-a798-420c-be5a-e6c6e230ebb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:41.227843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82c2e6-a798-420c-be5a-e6c6e230ebb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:41.228112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82c2e6-a798-420c-be5a-e6c6e230ebb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:41.237140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82c2e6-a798-420c-be5a-e6c6e230ebb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:41.238955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82c2e6-a798-420c-be5a-e6c6e230ebb4 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:41.243391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:41.253428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1105/2234] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:41.766023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90b5b5-d219-4c4d-a0a6-3e02755b7097 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:41.767481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7b90b5b5-d219-4c4d-a0a6-3e02755b7097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:41.788326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90b5b5-d219-4c4d-a0a6-3e02755b7097 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:41.794237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:41.795140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:41.795600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:41.796022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] PATH_INFO: `/v3/projects/867bb71226f94a928575748de9363fa0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:41.798072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=867bb71226f94a928575748de9363fa0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:41.807189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:41.860292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867bb71226f94a928575748de9363fa0', 'request_id': 'req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:41.867705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867bb71226f94a928575748de9363fa0', 'request_id': 'req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:41.874827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867bb71226f94a928575748de9363fa0', 'request_id': 'req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:41.890661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] The token cache is being invalidate because project 867bb71226f94a928575748de9363fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:41.890868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:41.891496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:41.897144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1130/2235] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:41 2022] DELETE /identity/v3/projects/867bb71226f94a928575748de9363fa0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:41.898822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8da176-4ae7-4e4e-b798-08bdf6563d45 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:41.899103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8da176-4ae7-4e4e-b798-08bdf6563d45 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:41.899390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8da176-4ae7-4e4e-b798-08bdf6563d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:41.921540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-df8da176-4ae7-4e4e-b798-08bdf6563d45 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:41.922880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-df8da176-4ae7-4e4e-b798-08bdf6563d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:41.996294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:41.997855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:42.040221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1106/2236] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:42.048278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b452eae-ff73-4b93-9cc8-b8b7de33067f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.048566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b452eae-ff73-4b93-9cc8-b8b7de33067f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.048839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b452eae-ff73-4b93-9cc8-b8b7de33067f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.071890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8b452eae-ff73-4b93-9cc8-b8b7de33067f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:42.072645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c0231-0a56-49c1-bcd4-c4e2f26b47cf admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.073097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8b452eae-ff73-4b93-9cc8-b8b7de33067f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:42.081514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1107/2237] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:42.081815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.082958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.083492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.083964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] PATH_INFO: `/v3/users/8d7d5e0f6ca341179d70c6ef426da35e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.085923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b452eae-ff73-4b93-9cc8-b8b7de33067f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.090647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7d5e0f6ca341179d70c6ef426da35e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.094330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.095252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b452eae-ff73-4b93-9cc8-b8b7de33067f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.100511 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.101274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.101426 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.101690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.102369 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:42.102669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.104285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.120989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1108/2238] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:39:42.121996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.134531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc544b-891f-4898-8878-5874abd3eea9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.140323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:42.142531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.143097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.143377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.144074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.144758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.146866 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.163100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7d5e0f6ca341179d70c6ef426da35e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:42.171763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7d5e0f6ca341179d70c6ef426da35e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:42.172447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7d5e0f6ca341179d70c6ef426da35e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:42.180148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1109/2239] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:42 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:42.184278 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.186124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:42.189172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7d5e0f6ca341179d70c6ef426da35e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:42.189978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-391cda26-ff3c-4289-bf6c-994a831b1f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7d5e0f6ca341179d70c6ef426da35e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:42.207253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1131/2240] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:41 2022] DELETE /identity/v3/users/8d7d5e0f6ca341179d70c6ef426da35e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:42.208827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3e9b-cd6d-421c-b0ba-d14bfcb6a956 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.209191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3e9b-cd6d-421c-b0ba-d14bfcb6a956 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.209561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3e9b-cd6d-421c-b0ba-d14bfcb6a956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.229562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c45e3e9b-cd6d-421c-b0ba-d14bfcb6a956 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:42.230966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c45e3e9b-cd6d-421c-b0ba-d14bfcb6a956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:42.269684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13426c26-7337-4161-b964-0713787ac3a1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.301072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1132/2241] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:42.312259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e3e9b-cd6d-421c-b0ba-d14bfcb6a956 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.313803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c45e3e9b-cd6d-421c-b0ba-d14bfcb6a956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:42.338632 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb76434-d898-4f57-accb-11469ef118f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.340115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb76434-d898-4f57-accb-11469ef118f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.340637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb76434-d898-4f57-accb-11469ef118f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.341111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb76434-d898-4f57-accb-11469ef118f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.342380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb76434-d898-4f57-accb-11469ef118f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.345722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb76434-d898-4f57-accb-11469ef118f0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.362238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:42.384225 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1110/2242] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:42 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:42.387145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb76434-d898-4f57-accb-11469ef118f0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.390117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e3e9b-cd6d-421c-b0ba-d14bfcb6a956 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.393996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12660897-fc5a-471f-b53d-0f50d2b0b626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.394609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12660897-fc5a-471f-b53d-0f50d2b0b626 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.394949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12660897-fc5a-471f-b53d-0f50d2b0b626 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.395282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12660897-fc5a-471f-b53d-0f50d2b0b626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.400416 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb76434-d898-4f57-accb-11469ef118f0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.406863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12660897-fc5a-471f-b53d-0f50d2b0b626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.408286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51780e-4c13-4e07-abeb-a59a58a32576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.408427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12660897-fc5a-471f-b53d-0f50d2b0b626 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.409076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51780e-4c13-4e07-abeb-a59a58a32576 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.410095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51780e-4c13-4e07-abeb-a59a58a32576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.410584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51780e-4c13-4e07-abeb-a59a58a32576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.411939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51780e-4c13-4e07-abeb-a59a58a32576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.414081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51780e-4c13-4e07-abeb-a59a58a32576 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.423208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1133/2243] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:39:42.440899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12660897-fc5a-471f-b53d-0f50d2b0b626 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.447365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1111/2244] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:39:42.466160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12660897-fc5a-471f-b53d-0f50d2b0b626 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.488299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89df931-9d40-4319-a1e7-56825859446b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.490169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89df931-9d40-4319-a1e7-56825859446b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.490618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89df931-9d40-4319-a1e7-56825859446b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.490889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89df931-9d40-4319-a1e7-56825859446b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.491548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89df931-9d40-4319-a1e7-56825859446b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.501551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89df931-9d40-4319-a1e7-56825859446b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.518522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1134/2245] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:42.522951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51780e-4c13-4e07-abeb-a59a58a32576 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.538556 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51780e-4c13-4e07-abeb-a59a58a32576 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.547326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1146160-1097-471e-a8ba-eecbe9ef41d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.548161 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1146160-1097-471e-a8ba-eecbe9ef41d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.548440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1146160-1097-471e-a8ba-eecbe9ef41d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.548812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1146160-1097-471e-a8ba-eecbe9ef41d0 admin admin] PATH_INFO: `/v3/projects/2158cd74bdb44c47a905e34e97c5807f/users/ccc10397cf6a4b9084bcd6cdb12b8607/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.582370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1146160-1097-471e-a8ba-eecbe9ef41d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2158cd74bdb44c47a905e34e97c5807f, user_id=ccc10397cf6a4b9084bcd6cdb12b8607, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.584697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1146160-1097-471e-a8ba-eecbe9ef41d0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.605267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1112/2246] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:42 2022] PUT /identity/v3/projects/2158cd74bdb44c47a905e34e97c5807f/users/ccc10397cf6a4b9084bcd6cdb12b8607/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:42.609512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecabf2-7e64-492e-bd16-40aa36d83bae None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.609797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecabf2-7e64-492e-bd16-40aa36d83bae None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.610070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecabf2-7e64-492e-bd16-40aa36d83bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.647336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-41ecabf2-7e64-492e-bd16-40aa36d83bae None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:42.648754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-41ecabf2-7e64-492e-bd16-40aa36d83bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:42.702701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:42.742191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1135/2247] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:42.753761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1146160-1097-471e-a8ba-eecbe9ef41d0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.867008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1146160-1097-471e-a8ba-eecbe9ef41d0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.877477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b4db-c983-47aa-9f14-b301bd2a7fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.878405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b4db-c983-47aa-9f14-b301bd2a7fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.878695 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b4db-c983-47aa-9f14-b301bd2a7fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.878946 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b4db-c983-47aa-9f14-b301bd2a7fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.895869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecabf2-7e64-492e-bd16-40aa36d83bae None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:42.898265 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-41ecabf2-7e64-492e-bd16-40aa36d83bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:42.913978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2b4db-c983-47aa-9f14-b301bd2a7fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.915434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2b4db-c983-47aa-9f14-b301bd2a7fe4 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:42.924808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecabf2-7e64-492e-bd16-40aa36d83bae None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:42.940837 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:42.943847 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:42.944264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:42.944635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] PATH_INFO: `/v3/projects/434dc77082bd461199e9ec803bb84716` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:42.949537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=434dc77082bd461199e9ec803bb84716)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:42.950355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1113/2248] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:42.952312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:43.042184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434dc77082bd461199e9ec803bb84716', 'request_id': 'req-85dad219-8591-488f-8007-f50a0927d722'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.051397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434dc77082bd461199e9ec803bb84716', 'request_id': 'req-85dad219-8591-488f-8007-f50a0927d722'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.061886 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434dc77082bd461199e9ec803bb84716', 'request_id': 'req-85dad219-8591-488f-8007-f50a0927d722'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.090087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] The token cache is being invalidate because project 434dc77082bd461199e9ec803bb84716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:43.091149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.092022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.097369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1136/2249] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:42 2022] DELETE /identity/v3/projects/434dc77082bd461199e9ec803bb84716 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:43.100170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b4db-c983-47aa-9f14-b301bd2a7fe4 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:43.113446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b4db-c983-47aa-9f14-b301bd2a7fe4 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:43.227578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:43.228294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:43.228640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:43.228974 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] PATH_INFO: `/v3/users/f4578ca2593142e8badc04cfd3faf70b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:43.230959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4578ca2593142e8badc04cfd3faf70b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:43.233874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:43.313811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4578ca2593142e8badc04cfd3faf70b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.325614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4578ca2593142e8badc04cfd3faf70b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.326815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4578ca2593142e8badc04cfd3faf70b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.337482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4578ca2593142e8badc04cfd3faf70b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.337862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:43.353145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4578ca2593142e8badc04cfd3faf70b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.372019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1114/2250] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:43 2022] DELETE /identity/v3/users/f4578ca2593142e8badc04cfd3faf70b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:43.696263 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:43.697612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:43.743076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dad219-8591-488f-8007-f50a0927d722 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:43.840038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:43.840895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:43.841311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:43.841821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] PATH_INFO: `/v3/projects/f59577ab879946c7954e6624de4eb6f0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:43.843246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f59577ab879946c7954e6624de4eb6f0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:43.846709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:43.932090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f59577ab879946c7954e6624de4eb6f0', 'request_id': 'req-d107aa2a-ad21-426e-8bec-0eb37a659ac3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.943915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f59577ab879946c7954e6624de4eb6f0', 'request_id': 'req-d107aa2a-ad21-426e-8bec-0eb37a659ac3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.953102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f59577ab879946c7954e6624de4eb6f0', 'request_id': 'req-d107aa2a-ad21-426e-8bec-0eb37a659ac3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.957035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:43.978544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] The token cache is being invalidate because project f59577ab879946c7954e6624de4eb6f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:43.978840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.979565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:43.982616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1137/2251] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:43 2022] DELETE /identity/v3/projects/f59577ab879946c7954e6624de4eb6f0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:43.986816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:43.989753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:44.008045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167e17e-30f6-482b-8233-14c6e9c0ec19 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:44.088409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:44.089160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:44.089525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:44.089863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] PATH_INFO: `/v3/users/89e0070c67b54fc5b40b80528bd8f533` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:44.091370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e0070c67b54fc5b40b80528bd8f533)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:44.093484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:44.157578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e0070c67b54fc5b40b80528bd8f533'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:44.167249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e0070c67b54fc5b40b80528bd8f533'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:44.168475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e0070c67b54fc5b40b80528bd8f533'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:44.179673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e0070c67b54fc5b40b80528bd8f533'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:44.208641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-24b16761-e6ee-4f56-bfcc-b76e7d685205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e0070c67b54fc5b40b80528bd8f533'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:44.226525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1115/2252] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:39:43 2022] DELETE /identity/v3/users/89e0070c67b54fc5b40b80528bd8f533 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:44.973498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:44.976108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:45.004390 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107aa2a-ad21-426e-8bec-0eb37a659ac3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:45.069950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:45.070588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.070948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.071564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] PATH_INFO: `/v3/projects/4c22cddd2942410985d2a8878e022af4` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.072480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c22cddd2942410985d2a8878e022af4)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:45.074552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:45.135048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c22cddd2942410985d2a8878e022af4', 'request_id': 'req-c3f74040-fdf4-43b8-bd37-1a962c6de62c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:45.141603 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c22cddd2942410985d2a8878e022af4', 'request_id': 'req-c3f74040-fdf4-43b8-bd37-1a962c6de62c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:45.149545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c22cddd2942410985d2a8878e022af4', 'request_id': 'req-c3f74040-fdf4-43b8-bd37-1a962c6de62c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:45.165512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] The token cache is being invalidate because project 4c22cddd2942410985d2a8878e022af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:39:45.166015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:45.169836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c3f74040-fdf4-43b8-bd37-1a962c6de62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:39:45.175169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1138/2253] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:39:44 2022] DELETE /identity/v3/projects/4c22cddd2942410985d2a8878e022af4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:45.177396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579edd8-d769-4aa0-80f7-dc12301504b7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.177858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579edd8-d769-4aa0-80f7-dc12301504b7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.178003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579edd8-d769-4aa0-80f7-dc12301504b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.204223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2579edd8-d769-4aa0-80f7-dc12301504b7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:45.205475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2579edd8-d769-4aa0-80f7-dc12301504b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:45.270523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:45.290776 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1116/2254] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:45.291322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f604d-c2e5-43b0-992c-97c4e3e6c82e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.291602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f604d-c2e5-43b0-992c-97c4e3e6c82e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.291871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f604d-c2e5-43b0-992c-97c4e3e6c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.320476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-157f604d-c2e5-43b0-992c-97c4e3e6c82e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:45.326621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-157f604d-c2e5-43b0-992c-97c4e3e6c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:45.349356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1139/2255] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:45.353552 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579edd8-d769-4aa0-80f7-dc12301504b7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:45.368295 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579edd8-d769-4aa0-80f7-dc12301504b7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:45.375718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:45.376341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.376645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.376880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.383109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:39:45.389555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:45.392147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:45.410343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1117/2256] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:39:45.414735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f604d-c2e5-43b0-992c-97c4e3e6c82e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:45.429453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f604d-c2e5-43b0-992c-97c4e3e6c82e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:45.441953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c1ac6-1905-4945-a85d-c40a81d03df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:45.443145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1ac6-1905-4945-a85d-c40a81d03df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.444405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1ac6-1905-4945-a85d-c40a81d03df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.444834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1ac6-1905-4945-a85d-c40a81d03df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.445725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c1ac6-1905-4945-a85d-c40a81d03df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:45.447986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c1ac6-1905-4945-a85d-c40a81d03df5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:45.490289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1140/2257] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:45 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:45.491467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:45.502392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00c7bf-8707-4013-a453-e1d45696a23c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:45.567920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8413-977d-4094-9b3c-aa873c8c5516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:45.568429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8413-977d-4094-9b3c-aa873c8c5516 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.568716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8413-977d-4094-9b3c-aa873c8c5516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.568980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8413-977d-4094-9b3c-aa873c8c5516 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.569631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea8413-977d-4094-9b3c-aa873c8c5516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:45.571580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea8413-977d-4094-9b3c-aa873c8c5516 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:45.593947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1118/2258] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:45 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:45.598150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c1ac6-1905-4945-a85d-c40a81d03df5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:45.608856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c1ac6-1905-4945-a85d-c40a81d03df5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:45.613799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8d63a-918a-45c8-a4fe-d6a615f39160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:45.614920 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8d63a-918a-45c8-a4fe-d6a615f39160 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.615195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8d63a-918a-45c8-a4fe-d6a615f39160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.615461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8d63a-918a-45c8-a4fe-d6a615f39160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.616130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8d63a-918a-45c8-a4fe-d6a615f39160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:45.617752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8d63a-918a-45c8-a4fe-d6a615f39160 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:45.631319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1141/2259] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:45.632797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8413-977d-4094-9b3c-aa873c8c5516 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:45.643209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8413-977d-4094-9b3c-aa873c8c5516 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:45.647997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d907b-97fd-488a-b48d-17ad1c433179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:45.648608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d907b-97fd-488a-b48d-17ad1c433179 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.648884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d907b-97fd-488a-b48d-17ad1c433179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.649150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d907b-97fd-488a-b48d-17ad1c433179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.649947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d907b-97fd-488a-b48d-17ad1c433179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:45.651675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d907b-97fd-488a-b48d-17ad1c433179 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:45.663847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1119/2260] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:39:45.668395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8d63a-918a-45c8-a4fe-d6a615f39160 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:45.680566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8d63a-918a-45c8-a4fe-d6a615f39160 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:45.684282 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:45.684942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.685768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.685768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] PATH_INFO: `/v3/projects/da601c9db023410d9a12c6d7bf1eb0e0/users/31da32c0efe44b9f8f8ada1c11fa8fb4/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.701813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da601c9db023410d9a12c6d7bf1eb0e0, user_id=31da32c0efe44b9f8f8ada1c11fa8fb4, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:45.704507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:45.718178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1142/2261] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:45 2022] PUT /identity/v3/projects/da601c9db023410d9a12c6d7bf1eb0e0/users/31da32c0efe44b9f8f8ada1c11fa8fb4/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:39:45.791280 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4747e-91b0-45d8-8974-68ddaf077d53 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:45.791758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4747e-91b0-45d8-8974-68ddaf077d53 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:45.792140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4747e-91b0-45d8-8974-68ddaf077d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:45.825600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ccd4747e-91b0-45d8-8974-68ddaf077d53 None None] MFA Rules not processed for user `31da32c0efe44b9f8f8ada1c11fa8fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:45.827019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ccd4747e-91b0-45d8-8974-68ddaf077d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:45.909487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1120/2262] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:45.916772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:45.918599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:45.944428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:46.009110 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09584193-0aef-4b83-a87c-f0ab0b16ba79 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:46.016604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f2fab-993d-4a9a-9789-d61645b6c483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:46.017262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2fab-993d-4a9a-9789-d61645b6c483 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:46.017568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2fab-993d-4a9a-9789-d61645b6c483 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:46.017843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2fab-993d-4a9a-9789-d61645b6c483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:46.028129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f2fab-993d-4a9a-9789-d61645b6c483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:46.029555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f2fab-993d-4a9a-9789-d61645b6c483 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:46.042302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1143/2263] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:47.744037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658c28-691d-4475-95d2-6a68c6d0d674 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:47.744562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658c28-691d-4475-95d2-6a68c6d0d674 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:47.744562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658c28-691d-4475-95d2-6a68c6d0d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:47.764395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ad658c28-691d-4475-95d2-6a68c6d0d674 None None] MFA Rules not processed for user `31da32c0efe44b9f8f8ada1c11fa8fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:47.765654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ad658c28-691d-4475-95d2-6a68c6d0d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:47.778323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1121/2264] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:48.962541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e1134-f9de-47c5-a3fa-ef374d89d2ff None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:48.963006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e1134-f9de-47c5-a3fa-ef374d89d2ff None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:48.969885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e1134-f9de-47c5-a3fa-ef374d89d2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:49.002380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-779e1134-f9de-47c5-a3fa-ef374d89d2ff None None] MFA Rules not processed for user `ccc10397cf6a4b9084bcd6cdb12b8607`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:49.003599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-779e1134-f9de-47c5-a3fa-ef374d89d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:49.157842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1144/2265] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:39:49.162395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658c28-691d-4475-95d2-6a68c6d0d674 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:49.164149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ad658c28-691d-4475-95d2-6a68c6d0d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:49.256041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:49.280949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658c28-691d-4475-95d2-6a68c6d0d674 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:49.288797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25899d82-15d4-42c9-8cd0-2adda579c773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:49.289563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25899d82-15d4-42c9-8cd0-2adda579c773 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:49.289853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25899d82-15d4-42c9-8cd0-2adda579c773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:49.290015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25899d82-15d4-42c9-8cd0-2adda579c773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:49.290742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25899d82-15d4-42c9-8cd0-2adda579c773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:49.292593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25899d82-15d4-42c9-8cd0-2adda579c773 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:49.328487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1122/2266] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:39:49 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:39:49.334730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e1134-f9de-47c5-a3fa-ef374d89d2ff None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:49.364532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e1134-f9de-47c5-a3fa-ef374d89d2ff None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:49.476016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d976-e9f7-475b-ac11-14bf9d23b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:49.477556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d976-e9f7-475b-ac11-14bf9d23b795 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:49.477932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d976-e9f7-475b-ac11-14bf9d23b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:49.478345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d976-e9f7-475b-ac11-14bf9d23b795 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:49.479178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5d976-e9f7-475b-ac11-14bf9d23b795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:49.481430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5d976-e9f7-475b-ac11-14bf9d23b795 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:49.512358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1145/2267] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:39:49 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:39:49.513491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25899d82-15d4-42c9-8cd0-2adda579c773 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:49.528780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25899d82-15d4-42c9-8cd0-2adda579c773 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:49.537942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf06237-c5fc-4168-a3e7-fcc45587b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:49.538576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf06237-c5fc-4168-a3e7-fcc45587b9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:49.538854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf06237-c5fc-4168-a3e7-fcc45587b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:49.539125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf06237-c5fc-4168-a3e7-fcc45587b9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:49.539781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf06237-c5fc-4168-a3e7-fcc45587b9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:49.547226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf06237-c5fc-4168-a3e7-fcc45587b9fa admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:49.569272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1123/2268] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:49.572564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d976-e9f7-475b-ac11-14bf9d23b795 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:49.587193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d976-e9f7-475b-ac11-14bf9d23b795 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:49.620289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048bf25-d8c0-4d51-a7ab-60744a1d648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:49.626428 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048bf25-d8c0-4d51-a7ab-60744a1d648a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:49.628380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048bf25-d8c0-4d51-a7ab-60744a1d648a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:49.629373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048bf25-d8c0-4d51-a7ab-60744a1d648a admin admin] PATH_INFO: `/v3/projects/c28df7f6904d4d5e9f1100742b99d6eb/users/4cf79e1933aa4ec2898aa0dec9dae6bc/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:49.698120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048bf25-d8c0-4d51-a7ab-60744a1d648a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28df7f6904d4d5e9f1100742b99d6eb, user_id=4cf79e1933aa4ec2898aa0dec9dae6bc, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:49.713247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048bf25-d8c0-4d51-a7ab-60744a1d648a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:49.752172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1146/2269] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:49 2022] PUT /identity/v3/projects/c28df7f6904d4d5e9f1100742b99d6eb/users/4cf79e1933aa4ec2898aa0dec9dae6bc/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:49.752995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf06237-c5fc-4168-a3e7-fcc45587b9fa admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:49.771459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf06237-c5fc-4168-a3e7-fcc45587b9fa admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:49.886091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca68ae-ae67-4e46-a45b-ae1e676d9f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:49.886814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca68ae-ae67-4e46-a45b-ae1e676d9f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:49.887441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca68ae-ae67-4e46-a45b-ae1e676d9f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:49.887772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca68ae-ae67-4e46-a45b-ae1e676d9f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:49.888432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca68ae-ae67-4e46-a45b-ae1e676d9f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:49.890231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca68ae-ae67-4e46-a45b-ae1e676d9f57 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:49.907411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1124/2270] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:39:49.911707 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048bf25-d8c0-4d51-a7ab-60744a1d648a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:49.925776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048bf25-d8c0-4d51-a7ab-60744a1d648a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:49.940009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d1b87-9dae-4b90-b130-fcf639477cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:49.940683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d1b87-9dae-4b90-b130-fcf639477cde admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:49.940956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d1b87-9dae-4b90-b130-fcf639477cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:49.941240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d1b87-9dae-4b90-b130-fcf639477cde admin admin] PATH_INFO: `/v3/projects/c28df7f6904d4d5e9f1100742b99d6eb/users/4cf79e1933aa4ec2898aa0dec9dae6bc/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:49.945915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d1b87-9dae-4b90-b130-fcf639477cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28df7f6904d4d5e9f1100742b99d6eb, user_id=4cf79e1933aa4ec2898aa0dec9dae6bc, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:49.952385 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d1b87-9dae-4b90-b130-fcf639477cde admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:49.980537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1147/2271] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:39:49 2022] PUT /identity/v3/projects/c28df7f6904d4d5e9f1100742b99d6eb/users/4cf79e1933aa4ec2898aa0dec9dae6bc/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:39:51.692743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8699a-7a63-402a-80c2-e8dfec6bef3a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:51.693636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8699a-7a63-402a-80c2-e8dfec6bef3a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:51.693917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8699a-7a63-402a-80c2-e8dfec6bef3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:51.724561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f9b8699a-7a63-402a-80c2-e8dfec6bef3a None None] MFA Rules not processed for user `a3d9e5dc719c430eac6b5c8d8b814566`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:51.727009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f9b8699a-7a63-402a-80c2-e8dfec6bef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:51.876942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1125/2272] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:53.809659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150aca1-1337-4e7b-9c9a-236683b99ccd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:53.810120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150aca1-1337-4e7b-9c9a-236683b99ccd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:53.810120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150aca1-1337-4e7b-9c9a-236683b99ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:53.817357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:53.836919 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4150aca1-1337-4e7b-9c9a-236683b99ccd None None] MFA Rules not processed for user `5573338f0bd64c729c7d89345069ca6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:53.838479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4150aca1-1337-4e7b-9c9a-236683b99ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:53.940036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1148/2273] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:58.056459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300477-ccd1-477c-93ad-6454f1291a6a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:58.057809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300477-ccd1-477c-93ad-6454f1291a6a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:58.058243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300477-ccd1-477c-93ad-6454f1291a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:58.125387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-03300477-ccd1-477c-93ad-6454f1291a6a None None] MFA Rules not processed for user `4cf79e1933aa4ec2898aa0dec9dae6bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:58.127487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-03300477-ccd1-477c-93ad-6454f1291a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:58.239611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:58.269545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1126/2274] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:58.286260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150aca1-1337-4e7b-9c9a-236683b99ccd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:58.291474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4150aca1-1337-4e7b-9c9a-236683b99ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:58.474925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150aca1-1337-4e7b-9c9a-236683b99ccd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:58.484288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce851cf0509947e783dbd5b4809bc897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:58.485458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:58.486017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:58.486454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:58.576358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:58.589124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:58.663831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1149/2275] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:39:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:39:58.880537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e44276-3e56-494d-a1f3-8a991bcd8ffa None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:58.880816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e44276-3e56-494d-a1f3-8a991bcd8ffa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:58.881091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e44276-3e56-494d-a1f3-8a991bcd8ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:58.909570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-01e44276-3e56-494d-a1f3-8a991bcd8ffa None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:39:58.911835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-01e44276-3e56-494d-a1f3-8a991bcd8ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:59.040474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1127/2276] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:39:59.102828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:39:59.104422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:39:59.244918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:39:59.260655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641015c-84d8-4b44-8a1c-8e6ad93776a1 service glance] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:39:59.265956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7840bc-7f07-4da3-8416-f135fc176055 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:39:59.266585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7840bc-7f07-4da3-8416-f135fc176055 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:39:59.266855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7840bc-7f07-4da3-8416-f135fc176055 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:39:59.267122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7840bc-7f07-4da3-8416-f135fc176055 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:39:59.284748 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7840bc-7f07-4da3-8416-f135fc176055 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:39:59.287106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7840bc-7f07-4da3-8416-f135fc176055 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:39:59.298492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1150/2277] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:39:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:40:09.033075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=30 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:40:09.294645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:40:09.458972 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68338876-faa8-4081-9ea1-9addfd1d5e6e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:09.459253 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68338876-faa8-4081-9ea1-9addfd1d5e6e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:09.459525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68338876-faa8-4081-9ea1-9addfd1d5e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:09.496440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-68338876-faa8-4081-9ea1-9addfd1d5e6e None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:09.498412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-68338876-faa8-4081-9ea1-9addfd1d5e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:09.522392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1128/2278] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:40:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:09.561832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7840bc-7f07-4da3-8416-f135fc176055 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:09.576451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7840bc-7f07-4da3-8416-f135fc176055 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:09.606275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89586d78-a94d-4964-a9f5-76d6af04c126 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:09.607162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89586d78-a94d-4964-a9f5-76d6af04c126 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:09.607647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89586d78-a94d-4964-a9f5-76d6af04c126 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:09.608138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89586d78-a94d-4964-a9f5-76d6af04c126 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:09.644070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89586d78-a94d-4964-a9f5-76d6af04c126 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:09.647592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89586d78-a94d-4964-a9f5-76d6af04c126 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:09.691689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1151/2279] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:40:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:40:10.713829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e6ed7-b9a9-4643-821c-22005f1ada03 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:10.714937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e6ed7-b9a9-4643-821c-22005f1ada03 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:10.715556 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e6ed7-b9a9-4643-821c-22005f1ada03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:10.737498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b89e6ed7-b9a9-4643-821c-22005f1ada03 None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:10.740475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b89e6ed7-b9a9-4643-821c-22005f1ada03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:10.756000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1129/2280] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:40:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:10.769433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89586d78-a94d-4964-a9f5-76d6af04c126 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:10.807009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89586d78-a94d-4964-a9f5-76d6af04c126 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:10.816832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed5dd1-60f1-4f55-921f-7656a038d560 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:10.817523 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5dd1-60f1-4f55-921f-7656a038d560 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:10.817804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5dd1-60f1-4f55-921f-7656a038d560 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:10.818078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5dd1-60f1-4f55-921f-7656a038d560 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:10.829548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed5dd1-60f1-4f55-921f-7656a038d560 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:10.831488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed5dd1-60f1-4f55-921f-7656a038d560 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:10.845984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1152/2281] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:40:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:40:12.196803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac778ac-5c0c-4d32-9c2e-0a82a040e812 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:12.197421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac778ac-5c0c-4d32-9c2e-0a82a040e812 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:12.197421 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac778ac-5c0c-4d32-9c2e-0a82a040e812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:12.206327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8ede-18d0-42d7-82b6-0915e3e394de None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:12.207529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8ede-18d0-42d7-82b6-0915e3e394de None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:12.208033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8ede-18d0-42d7-82b6-0915e3e394de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:12.241318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-eac778ac-5c0c-4d32-9c2e-0a82a040e812 None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:12.242825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eac778ac-5c0c-4d32-9c2e-0a82a040e812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:12.243792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d9fb8ede-18d0-42d7-82b6-0915e3e394de None None] MFA Rules not processed for user `ccc10397cf6a4b9084bcd6cdb12b8607`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:12.245159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d9fb8ede-18d0-42d7-82b6-0915e3e394de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:12.252730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1130/2282] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 10 13:40:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:12.260122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1153/2283] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:40:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:40:12.277899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac778ac-5c0c-4d32-9c2e-0a82a040e812 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:12.300257 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac778ac-5c0c-4d32-9c2e-0a82a040e812 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:12.307288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740a2e1-e8ee-47de-985e-7a7a486ad19c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:12.308246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740a2e1-e8ee-47de-985e-7a7a486ad19c service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:12.308778 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740a2e1-e8ee-47de-985e-7a7a486ad19c service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:12.309289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740a2e1-e8ee-47de-985e-7a7a486ad19c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:12.322504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740a2e1-e8ee-47de-985e-7a7a486ad19c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:12.324796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740a2e1-e8ee-47de-985e-7a7a486ad19c service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:12.335310 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1131/2284] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:40:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:40:12.788806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40342004-a8d3-4037-b1f3-456a333ea14b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:12.789438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40342004-a8d3-4037-b1f3-456a333ea14b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:12.789877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40342004-a8d3-4037-b1f3-456a333ea14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:12.819407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-40342004-a8d3-4037-b1f3-456a333ea14b None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:12.820922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-40342004-a8d3-4037-b1f3-456a333ea14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:12.833646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1154/2285] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:40:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:12.848940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740a2e1-e8ee-47de-985e-7a7a486ad19c service swift] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:12.866716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740a2e1-e8ee-47de-985e-7a7a486ad19c service swift] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:12.882496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ddda0-f431-421f-ad0b-ace7548bbedc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:12.883532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ddda0-f431-421f-ad0b-ace7548bbedc service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:12.884049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ddda0-f431-421f-ad0b-ace7548bbedc service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:12.884403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ddda0-f431-421f-ad0b-ace7548bbedc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:12.906176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ddda0-f431-421f-ad0b-ace7548bbedc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:12.908134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ddda0-f431-421f-ad0b-ace7548bbedc service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:12.927048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1132/2286] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:40:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:40:13.506051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa3f32-7d72-40fc-800b-788a0fcd59d2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:13.506564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa3f32-7d72-40fc-800b-788a0fcd59d2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:13.506667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa3f32-7d72-40fc-800b-788a0fcd59d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:13.545274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-25aa3f32-7d72-40fc-800b-788a0fcd59d2 None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:13.547355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-25aa3f32-7d72-40fc-800b-788a0fcd59d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:13.563604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1155/2287] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 10 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:13.571144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ddda0-f431-421f-ad0b-ace7548bbedc service swift] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:13.585289 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ddda0-f431-421f-ad0b-ace7548bbedc service swift] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:13.591236 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9c2cf-c105-4b11-b040-adb3f16e6174 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:13.592097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9c2cf-c105-4b11-b040-adb3f16e6174 service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:13.592577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9c2cf-c105-4b11-b040-adb3f16e6174 service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:13.593026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9c2cf-c105-4b11-b040-adb3f16e6174 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:13.603515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9c2cf-c105-4b11-b040-adb3f16e6174 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:13.605693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9c2cf-c105-4b11-b040-adb3f16e6174 service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:13.615941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1133/2288] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:40:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:40:16.416354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b876ec9-5009-444c-9dfc-2c4f2111caf8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:16.416960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b876ec9-5009-444c-9dfc-2c4f2111caf8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:16.417068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b876ec9-5009-444c-9dfc-2c4f2111caf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:16.451529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0b876ec9-5009-444c-9dfc-2c4f2111caf8 None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:16.452752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0b876ec9-5009-444c-9dfc-2c4f2111caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:16.470990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1156/2289] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:40:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:16.533603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9c2cf-c105-4b11-b040-adb3f16e6174 service swift] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:16.549496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9c2cf-c105-4b11-b040-adb3f16e6174 service swift] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:16.560787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36191d08-0f9a-41b7-92b4-a65f77f9655c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:16.573895 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191d08-0f9a-41b7-92b4-a65f77f9655c service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:16.574454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191d08-0f9a-41b7-92b4-a65f77f9655c service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:16.574877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191d08-0f9a-41b7-92b4-a65f77f9655c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:16.585290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36191d08-0f9a-41b7-92b4-a65f77f9655c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:16.586811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36191d08-0f9a-41b7-92b4-a65f77f9655c service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:16.595920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1134/2290] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:40:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:40:16.867864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b876ec9-5009-444c-9dfc-2c4f2111caf8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:16.982972 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b876ec9-5009-444c-9dfc-2c4f2111caf8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:16.989790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:16.990827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:16.991247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:16.991648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] PATH_INFO: `/v3/users/31da32c0efe44b9f8f8ada1c11fa8fb4` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:16.993283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=31da32c0efe44b9f8f8ada1c11fa8fb4)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:16.996308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:17.132310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31da32c0efe44b9f8f8ada1c11fa8fb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:17.156890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31da32c0efe44b9f8f8ada1c11fa8fb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:17.157885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31da32c0efe44b9f8f8ada1c11fa8fb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:17.181902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31da32c0efe44b9f8f8ada1c11fa8fb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:17.182945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31da32c0efe44b9f8f8ada1c11fa8fb4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:17.198463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1157/2291] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:40:16 2022] DELETE /identity/v3/users/31da32c0efe44b9f8f8ada1c11fa8fb4 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:40:17.202910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfd66-61ac-429b-9d9f-27e2a8b807fe None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:17.203569 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfd66-61ac-429b-9d9f-27e2a8b807fe None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:17.204554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfd66-61ac-429b-9d9f-27e2a8b807fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:17.231304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f78cfd66-61ac-429b-9d9f-27e2a8b807fe None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:17.234314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f78cfd66-61ac-429b-9d9f-27e2a8b807fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:17.337375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1135/2292] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:17.365036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:17.368484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:17.476895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:40:17.569544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b72a60-cd00-49e0-96da-2380dffd0dca admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:17.584441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16d5c9-6d84-4248-8ecc-4b90aa0abcf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:17.585519 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d5c9-6d84-4248-8ecc-4b90aa0abcf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:17.586044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d5c9-6d84-4248-8ecc-4b90aa0abcf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:17.586394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d5c9-6d84-4248-8ecc-4b90aa0abcf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:17.619534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16d5c9-6d84-4248-8ecc-4b90aa0abcf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:17.621275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16d5c9-6d84-4248-8ecc-4b90aa0abcf2 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:17.647521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1158/2293] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:40:18.253184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cfd66-61ac-429b-9d9f-27e2a8b807fe None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:18.254781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f78cfd66-61ac-429b-9d9f-27e2a8b807fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:18.286469 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cfd66-61ac-429b-9d9f-27e2a8b807fe None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:18.305700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:18.306393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:18.306698 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:18.307041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] PATH_INFO: `/v3/projects/da601c9db023410d9a12c6d7bf1eb0e0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:18.310089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da601c9db023410d9a12c6d7bf1eb0e0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:18.312442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:18.474899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da601c9db023410d9a12c6d7bf1eb0e0', 'request_id': 'req-dc745975-6151-46dc-a788-0b5e11b74057'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:18.494112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da601c9db023410d9a12c6d7bf1eb0e0', 'request_id': 'req-dc745975-6151-46dc-a788-0b5e11b74057'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:18.507985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da601c9db023410d9a12c6d7bf1eb0e0', 'request_id': 'req-dc745975-6151-46dc-a788-0b5e11b74057'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:18.533969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] The token cache is being invalidate because project da601c9db023410d9a12c6d7bf1eb0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:40:18.534306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:18.534936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc745975-6151-46dc-a788-0b5e11b74057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:40:18.537123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1136/2294] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:40:18 2022] DELETE /identity/v3/projects/da601c9db023410d9a12c6d7bf1eb0e0 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:40:27.633415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=14 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:40:28.530488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:40:37.266646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa7ed0-0c5e-492e-aa27-2484e1febc7b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.267268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa7ed0-0c5e-492e-aa27-2484e1febc7b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.267268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa7ed0-0c5e-492e-aa27-2484e1febc7b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.268581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1159/2295] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:40:37.276216 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a2638-665d-4eca-8af9-f5a17685b868 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.276796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a2638-665d-4eca-8af9-f5a17685b868 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.276796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a2638-665d-4eca-8af9-f5a17685b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.296250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4d1a2638-665d-4eca-8af9-f5a17685b868 None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:37.297501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4d1a2638-665d-4eca-8af9-f5a17685b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:37.367996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1137/2296] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:40:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:37.382281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa7ed0-0c5e-492e-aa27-2484e1febc7b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:37.383570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-05fa7ed0-0c5e-492e-aa27-2484e1febc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:37.442672 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa7ed0-0c5e-492e-aa27-2484e1febc7b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:37.446176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619338bb-5919-4852-9bd1-c2a8a107a691 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce851cf0509947e783dbd5b4809bc897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:37.446880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619338bb-5919-4852-9bd1-c2a8a107a691 service glance] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.447050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619338bb-5919-4852-9bd1-c2a8a107a691 service glance] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.447300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619338bb-5919-4852-9bd1-c2a8a107a691 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.457707 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619338bb-5919-4852-9bd1-c2a8a107a691 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:37.459142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619338bb-5919-4852-9bd1-c2a8a107a691 service glance] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:37.470465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1160/2297] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:40:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:40:37.544399 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fee299-d9b3-4ed9-90ff-b710fc2551a3 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.544685 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fee299-d9b3-4ed9-90ff-b710fc2551a3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.544954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fee299-d9b3-4ed9-90ff-b710fc2551a3 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.546652 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1138/2298] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:40:37.551356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737eab-c860-4d6e-820a-096e1e4f2091 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.551642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737eab-c860-4d6e-820a-096e1e4f2091 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.551916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737eab-c860-4d6e-820a-096e1e4f2091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.584076 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-70737eab-c860-4d6e-820a-096e1e4f2091 None None] MFA Rules not processed for user `234eed198ab84e95ae1a7f03785a1481`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:37.585802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-70737eab-c860-4d6e-820a-096e1e4f2091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:37.656075 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1161/2299] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:40:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:37.672479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fee299-d9b3-4ed9-90ff-b710fc2551a3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:37.678389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-17fee299-d9b3-4ed9-90ff-b710fc2551a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:37.701316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fee299-d9b3-4ed9-90ff-b710fc2551a3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:37.704496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882fc32-b82f-48a3-aa3b-99554909e8cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:37.705047 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882fc32-b82f-48a3-aa3b-99554909e8cf service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.705517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882fc32-b82f-48a3-aa3b-99554909e8cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.705796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882fc32-b82f-48a3-aa3b-99554909e8cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.714869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882fc32-b82f-48a3-aa3b-99554909e8cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:37.716629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882fc32-b82f-48a3-aa3b-99554909e8cf service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:37.725530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1139/2300] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:40:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:40:37.840730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d07dc-5a10-46fc-ace9-c2f477a5c269 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.841008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d07dc-5a10-46fc-ace9-c2f477a5c269 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.841300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d07dc-5a10-46fc-ace9-c2f477a5c269 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.842645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1162/2301] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:40:37.848851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a510d1-92a0-4cc8-ba0f-eb568fc90ede None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.849128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a510d1-92a0-4cc8-ba0f-eb568fc90ede None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.849449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a510d1-92a0-4cc8-ba0f-eb568fc90ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.870822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-38a510d1-92a0-4cc8-ba0f-eb568fc90ede None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:37.872331 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-38a510d1-92a0-4cc8-ba0f-eb568fc90ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:37.881715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1140/2302] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:40:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:37.891950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d07dc-5a10-46fc-ace9-c2f477a5c269 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:37.903168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d07dc-5a10-46fc-ace9-c2f477a5c269 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:37.906662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdf91f-238c-4c66-9d00-d4bbe8a452be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:37.907245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdf91f-238c-4c66-9d00-d4bbe8a452be service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:37.907522 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdf91f-238c-4c66-9d00-d4bbe8a452be service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:37.907790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdf91f-238c-4c66-9d00-d4bbe8a452be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:37.918118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdf91f-238c-4c66-9d00-d4bbe8a452be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:37.919498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdf91f-238c-4c66-9d00-d4bbe8a452be service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:37.930001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1163/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:40:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:40:39.880454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a510d1-92a0-4cc8-ba0f-eb568fc90ede None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:39.881460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-38a510d1-92a0-4cc8-ba0f-eb568fc90ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:39.957137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a510d1-92a0-4cc8-ba0f-eb568fc90ede None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:39.962252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:39.962861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:39.963138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:39.963405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:39.965171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:39.978515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:39.980168 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:39.990443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1141/2304] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:40:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:40:42.786065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746deec2-39e0-4f7c-a482-b23d5dd139cd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:42.786556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746deec2-39e0-4f7c-a482-b23d5dd139cd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:42.786655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746deec2-39e0-4f7c-a482-b23d5dd139cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:42.805364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-746deec2-39e0-4f7c-a482-b23d5dd139cd None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:40:42.806539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-746deec2-39e0-4f7c-a482-b23d5dd139cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:42.860429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1164/2305] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:40:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:40:42.873928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:40:42.875529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:40:42.942113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:40:42.946681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed52a5d-a35b-44de-b8b5-e8fdb8a20910 service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:40:42.951947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e203fa9-f84c-4964-9e20-0cc14cfb8356 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:40:42.952550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e203fa9-f84c-4964-9e20-0cc14cfb8356 service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:40:42.952822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e203fa9-f84c-4964-9e20-0cc14cfb8356 service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:40:42.953197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e203fa9-f84c-4964-9e20-0cc14cfb8356 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:40:42.962024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e203fa9-f84c-4964-9e20-0cc14cfb8356 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:40:42.963856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e203fa9-f84c-4964-9e20-0cc14cfb8356 service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:40:42.972458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1142/2306] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:40:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:40:52.853757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=80 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:40:52.968877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:41:00.782948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746deec2-39e0-4f7c-a482-b23d5dd139cd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:41:00.792789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746deec2-39e0-4f7c-a482-b23d5dd139cd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:41:00.796266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:41:00.797872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:00.798882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:00.800378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:00.802139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:00.863699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:41:00.865124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:41:00.874809 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1165/2307] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:41:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:41:08.593750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711dc83-7886-499a-9272-c988c1470865 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:08.594255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711dc83-7886-499a-9272-c988c1470865 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:08.594364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711dc83-7886-499a-9272-c988c1470865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:08.613906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f711dc83-7886-499a-9272-c988c1470865 None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:41:08.615554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f711dc83-7886-499a-9272-c988c1470865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:08.623733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1143/2308] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 10 13:41:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:41:08.648741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:41:08.659948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a77b3c-1538-49ac-bc88-ce6d44fe9531 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:41:08.664815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae11e3-0010-402d-824a-cc9d1682b6bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:41:08.665431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae11e3-0010-402d-824a-cc9d1682b6bf service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:08.665710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae11e3-0010-402d-824a-cc9d1682b6bf service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:08.665980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae11e3-0010-402d-824a-cc9d1682b6bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:08.673832 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae11e3-0010-402d-824a-cc9d1682b6bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:41:08.675966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae11e3-0010-402d-824a-cc9d1682b6bf service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:41:08.683850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1166/2309] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:41:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:41:18.608140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:41:18.680617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=36 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:41:24.247049 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d2d23-22e2-49bc-8447-c14c8b6e5b44 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:24.247551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d2d23-22e2-49bc-8447-c14c8b6e5b44 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:24.247645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d2d23-22e2-49bc-8447-c14c8b6e5b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:24.266310 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-353d2d23-22e2-49bc-8447-c14c8b6e5b44 None None] MFA Rules not processed for user `a3d9e5dc719c430eac6b5c8d8b814566`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:41:24.267633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-353d2d23-22e2-49bc-8447-c14c8b6e5b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:24.323251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1144/2310] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:41:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:41:33.239964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8760a32-a945-4d41-9a85-73d39d965383 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:41:33.240610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8760a32-a945-4d41-9a85-73d39d965383 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:41:33.240610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8760a32-a945-4d41-9a85-73d39d965383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:41:33.262546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b8760a32-a945-4d41-9a85-73d39d965383 None None] MFA Rules not processed for user `5573338f0bd64c729c7d89345069ca6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:41:33.263761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b8760a32-a945-4d41-9a85-73d39d965383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:41:33.326623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1167/2311] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:41:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:41:34.316668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=25 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:41:43.320888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=25 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:06.866246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc92e-ff89-4982-8df8-6c46e53a7a88 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:06.866795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc92e-ff89-4982-8df8-6c46e53a7a88 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:06.866795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc92e-ff89-4982-8df8-6c46e53a7a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:06.890899 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a7dbc92e-ff89-4982-8df8-6c46e53a7a88 None None] MFA Rules not processed for user `411ff3f0306848ee91108343982806ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:06.893138 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a7dbc92e-ff89-4982-8df8-6c46e53a7a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:06.966359 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1145/2312] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:07.746263 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b40623-b6f9-4331-887b-3399aa4674bc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:07.746798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b40623-b6f9-4331-887b-3399aa4674bc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:07.746798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b40623-b6f9-4331-887b-3399aa4674bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:07.767754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-15b40623-b6f9-4331-887b-3399aa4674bc None None] MFA Rules not processed for user `9cecae4323404bd29fe8c05a704de273`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:07.768999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-15b40623-b6f9-4331-887b-3399aa4674bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:07.824715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1168/2313] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:16.957951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=25 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:17.818176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=25 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:33.081290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbc92e-ff89-4982-8df8-6c46e53a7a88 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:33.092818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbc92e-ff89-4982-8df8-6c46e53a7a88 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:33.097419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:33.098038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:33.098324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:33.098587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:33.100350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:33.128234 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:33.130681 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:33.145945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1146/2314] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:33.942152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b40623-b6f9-4331-887b-3399aa4674bc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:33.943553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-15b40623-b6f9-4331-887b-3399aa4674bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:34.015921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b40623-b6f9-4331-887b-3399aa4674bc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:34.020347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:34.020971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:34.021265 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:34.021538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:34.023986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:34.038204 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:34.039549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:34.050961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1169/2315] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:43.139279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:44.045459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=26 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:46.163828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:46.165994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:46.184442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2332dc8-0e76-463b-9e2c-34f0ac01fd19 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:46.191996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b01da7-e6d2-461d-a0f1-fa88b6af0849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:46.192751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01da7-e6d2-461d-a0f1-fa88b6af0849 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:46.193115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01da7-e6d2-461d-a0f1-fa88b6af0849 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:46.193435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01da7-e6d2-461d-a0f1-fa88b6af0849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:46.195658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c5b01da7-e6d2-461d-a0f1-fa88b6af0849 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:46.210939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b01da7-e6d2-461d-a0f1-fa88b6af0849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:46.212820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b01da7-e6d2-461d-a0f1-fa88b6af0849 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:46.225075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1147/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:46.459919 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:46.461966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:46.478959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5edbd-89f7-4531-a130-3ffa20bace99 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:46.484635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f960c-b917-4dc7-bde4-87fc788fdc05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:46.485683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f960c-b917-4dc7-bde4-87fc788fdc05 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:46.486089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f960c-b917-4dc7-bde4-87fc788fdc05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:46.486427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f960c-b917-4dc7-bde4-87fc788fdc05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:46.488737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-779f960c-b917-4dc7-bde4-87fc788fdc05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:46.621114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f960c-b917-4dc7-bde4-87fc788fdc05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:46.627801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f960c-b917-4dc7-bde4-87fc788fdc05 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:46.645973 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1170/2317] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:50.021559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682df23e-4cf8-4f3f-b722-1d5115c6b5d8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:50.022123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682df23e-4cf8-4f3f-b722-1d5115c6b5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:50.022691 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682df23e-4cf8-4f3f-b722-1d5115c6b5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:50.044190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-682df23e-4cf8-4f3f-b722-1d5115c6b5d8 None None] MFA Rules not processed for user `a3d9e5dc719c430eac6b5c8d8b814566`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:50.045441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-682df23e-4cf8-4f3f-b722-1d5115c6b5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:50.052831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1148/2318] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:52.587733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a8750-35e3-4f4c-a4df-35b9b41d7d4f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:52.588295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a8750-35e3-4f4c-a4df-35b9b41d7d4f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:52.588295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a8750-35e3-4f4c-a4df-35b9b41d7d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:52.621448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-228a8750-35e3-4f4c-a4df-35b9b41d7d4f None None] MFA Rules not processed for user `289c7d8b89cd445a924f5ac07e85fe48`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:52.623266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-228a8750-35e3-4f4c-a4df-35b9b41d7d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:52.733492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1171/2319] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:53.351851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682df23e-4cf8-4f3f-b722-1d5115c6b5d8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:53.377517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682df23e-4cf8-4f3f-b722-1d5115c6b5d8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:53.383371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:53.384573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:53.384999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:53.385714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:53.400025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:53.432598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:53.434909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:53.448788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1149/2320] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:54.245352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a8750-35e3-4f4c-a4df-35b9b41d7d4f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:54.249772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-228a8750-35e3-4f4c-a4df-35b9b41d7d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:54.254812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:54.267441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a8750-35e3-4f4c-a4df-35b9b41d7d4f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:54.276952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:54.277482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:54.277788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:54.278083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:54.279932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:54.282584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51360ba5-8be4-4e6c-a8e2-95e08ad40f8a service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:54.293539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:54.295318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:54.295827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:54.296462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:54.297513 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:54.297998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:54.300385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:54.307251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1172/2321] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:54.313570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:54.314971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:54.316376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:54.318315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:54.332603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1150/2322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:42:54.336098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d4528-9059-48b5-8a35-7bd7f971f8c5 service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:54.342296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:54.343088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:54.343622 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:54.344006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] PATH_INFO: `/v3/users/5573338f0bd64c729c7d89345069ca6f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:54.345261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5573338f0bd64c729c7d89345069ca6f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:54.347449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:54.398044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5573338f0bd64c729c7d89345069ca6f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.405906 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5573338f0bd64c729c7d89345069ca6f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.406530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5573338f0bd64c729c7d89345069ca6f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.420680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5573338f0bd64c729c7d89345069ca6f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.421432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-efacd786-a0f5-45f9-b3c4-5a59f59f9242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5573338f0bd64c729c7d89345069ca6f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.430231 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:54.436965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1173/2323] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:42:54 2022] DELETE /identity/v3/users/5573338f0bd64c729c7d89345069ca6f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:42:54.628173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:54.640323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a922728-af35-41f0-85b7-29ecb013d450 service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:54.694837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:54.695443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:54.695716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:54.695978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] PATH_INFO: `/v3/projects/d8eda7c7d6a348e08e81a6b2496b09e5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:54.697045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8eda7c7d6a348e08e81a6b2496b09e5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:54.698972 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:54.745759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8eda7c7d6a348e08e81a6b2496b09e5', 'request_id': 'req-dc951412-eb2e-4d09-b673-2a09bf837a79'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.751411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8eda7c7d6a348e08e81a6b2496b09e5', 'request_id': 'req-dc951412-eb2e-4d09-b673-2a09bf837a79'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.757894 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8eda7c7d6a348e08e81a6b2496b09e5', 'request_id': 'req-dc951412-eb2e-4d09-b673-2a09bf837a79'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.772983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] The token cache is being invalidate because project d8eda7c7d6a348e08e81a6b2496b09e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:42:54.773299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.773931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dc951412-eb2e-4d09-b673-2a09bf837a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:54.775409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1151/2324] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:42:54 2022] DELETE /identity/v3/projects/d8eda7c7d6a348e08e81a6b2496b09e5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:54.781087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6af9-a9d4-4560-8c89-9a4b8a4a9076 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:54.781395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6af9-a9d4-4560-8c89-9a4b8a4a9076 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:54.781667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6af9-a9d4-4560-8c89-9a4b8a4a9076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:54.800644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c19b6af9-a9d4-4560-8c89-9a4b8a4a9076 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:54.801876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c19b6af9-a9d4-4560-8c89-9a4b8a4a9076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:54.870516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1174/2325] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:54.878245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf0e3-8647-41ec-8975-30c4ae32d205 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:54.878530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf0e3-8647-41ec-8975-30c4ae32d205 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:54.878799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf0e3-8647-41ec-8975-30c4ae32d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:54.899991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d1fbf0e3-8647-41ec-8975-30c4ae32d205 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:54.901181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d1fbf0e3-8647-41ec-8975-30c4ae32d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:54.910712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1152/2326] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:54.915069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b6af9-a9d4-4560-8c89-9a4b8a4a9076 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:54.925994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b6af9-a9d4-4560-8c89-9a4b8a4a9076 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:54.930743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:54.931389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:54.931674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:54.932380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:54.933078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:42:54.933404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:54.935340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:54.951537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1175/2327] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:42:54.953824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbf0e3-8647-41ec-8975-30c4ae32d205 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:54.963351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbf0e3-8647-41ec-8975-30c4ae32d205 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:54.968206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d8b38-fcdf-4ff4-af4b-7a68923d0f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:54.968814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d8b38-fcdf-4ff4-af4b-7a68923d0f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:54.969089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d8b38-fcdf-4ff4-af4b-7a68923d0f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:54.969401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d8b38-fcdf-4ff4-af4b-7a68923d0f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:54.970020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d8b38-fcdf-4ff4-af4b-7a68923d0f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:54.971925 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d8b38-fcdf-4ff4-af4b-7a68923d0f3f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.009541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1153/2328] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:42:54 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:42:55.011038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.022194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45848e7c-fa02-4bf7-a410-197789a7b02d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.076486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88bcf5-51a9-4835-bf58-d1a383a5bb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.077756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88bcf5-51a9-4835-bf58-d1a383a5bb3a admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.078039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88bcf5-51a9-4835-bf58-d1a383a5bb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.078307 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88bcf5-51a9-4835-bf58-d1a383a5bb3a admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.078924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88bcf5-51a9-4835-bf58-d1a383a5bb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.080841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88bcf5-51a9-4835-bf58-d1a383a5bb3a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.103516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1176/2329] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:42:55 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:42:55.107726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d8b38-fcdf-4ff4-af4b-7a68923d0f3f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.118159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d8b38-fcdf-4ff4-af4b-7a68923d0f3f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.122723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d9e9e-64d3-4663-be7e-b76af9383d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.123326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9e9e-64d3-4663-be7e-b76af9383d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.123598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9e9e-64d3-4663-be7e-b76af9383d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.123862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d9e9e-64d3-4663-be7e-b76af9383d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.125003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d9e9e-64d3-4663-be7e-b76af9383d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.126609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d9e9e-64d3-4663-be7e-b76af9383d6d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.136932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1154/2330] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:42:55.141288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88bcf5-51a9-4835-bf58-d1a383a5bb3a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.151293 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88bcf5-51a9-4835-bf58-d1a383a5bb3a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.156188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86264b4d-89db-4e32-ab66-c16608a0d133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.156797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86264b4d-89db-4e32-ab66-c16608a0d133 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.157081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86264b4d-89db-4e32-ab66-c16608a0d133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.157392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86264b4d-89db-4e32-ab66-c16608a0d133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.158039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86264b4d-89db-4e32-ab66-c16608a0d133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.159628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86264b4d-89db-4e32-ab66-c16608a0d133 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.170700 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1177/2331] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:42:55.174916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d9e9e-64d3-4663-be7e-b76af9383d6d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.181115 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:55.185254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d9e9e-64d3-4663-be7e-b76af9383d6d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.191692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24663e6e-6302-4eab-a57a-0b54ddb39791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.192527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663e6e-6302-4eab-a57a-0b54ddb39791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.192996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663e6e-6302-4eab-a57a-0b54ddb39791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.193547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663e6e-6302-4eab-a57a-0b54ddb39791 admin admin] PATH_INFO: `/v3/projects/7fdeef4dbd0c43f1a27216b3a12ee99e/users/fc29b2649cb147f0a4ce06d1b5f2da61/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.354210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24663e6e-6302-4eab-a57a-0b54ddb39791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fdeef4dbd0c43f1a27216b3a12ee99e, user_id=fc29b2649cb147f0a4ce06d1b5f2da61, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.356921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24663e6e-6302-4eab-a57a-0b54ddb39791 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.374057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1155/2332] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:42:55 2022] PUT /identity/v3/projects/7fdeef4dbd0c43f1a27216b3a12ee99e/users/fc29b2649cb147f0a4ce06d1b5f2da61/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:55.385496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7cd0-410a-42d6-97cb-4278a6af5f3f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.385998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7cd0-410a-42d6-97cb-4278a6af5f3f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.386114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7cd0-410a-42d6-97cb-4278a6af5f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.416193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5d8c7cd0-410a-42d6-97cb-4278a6af5f3f None None] MFA Rules not processed for user `fc29b2649cb147f0a4ce06d1b5f2da61`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:55.417533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5d8c7cd0-410a-42d6-97cb-4278a6af5f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:55.482880 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:55.495655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1178/2333] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:55.499964 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24663e6e-6302-4eab-a57a-0b54ddb39791 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.571497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24663e6e-6302-4eab-a57a-0b54ddb39791 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.577715 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40984c-fa51-4fb3-92a2-bcf168816f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.579247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40984c-fa51-4fb3-92a2-bcf168816f5d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.579699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40984c-fa51-4fb3-92a2-bcf168816f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.579971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40984c-fa51-4fb3-92a2-bcf168816f5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.580593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40984c-fa51-4fb3-92a2-bcf168816f5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.582544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40984c-fa51-4fb3-92a2-bcf168816f5d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.633205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1156/2334] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:42:55 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:42:55.637486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c7cd0-410a-42d6-97cb-4278a6af5f3f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.648359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c7cd0-410a-42d6-97cb-4278a6af5f3f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.707892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bd9a5-c71a-4d98-9e7a-cb9aedcbcc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.708506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bd9a5-c71a-4d98-9e7a-cb9aedcbcc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.708783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bd9a5-c71a-4d98-9e7a-cb9aedcbcc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.709051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bd9a5-c71a-4d98-9e7a-cb9aedcbcc58 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.709705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bd9a5-c71a-4d98-9e7a-cb9aedcbcc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.711690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bd9a5-c71a-4d98-9e7a-cb9aedcbcc58 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.734118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1179/2335] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:42:55 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:42:55.738473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40984c-fa51-4fb3-92a2-bcf168816f5d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.749764 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40984c-fa51-4fb3-92a2-bcf168816f5d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.754323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55034095-9188-4724-9bd4-8d5e723aaef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.760435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55034095-9188-4724-9bd4-8d5e723aaef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.760712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55034095-9188-4724-9bd4-8d5e723aaef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.760979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55034095-9188-4724-9bd4-8d5e723aaef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.761656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55034095-9188-4724-9bd4-8d5e723aaef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.763271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55034095-9188-4724-9bd4-8d5e723aaef0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.773423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1157/2336] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:42:55.777905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bd9a5-c71a-4d98-9e7a-cb9aedcbcc58 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.788068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bd9a5-c71a-4d98-9e7a-cb9aedcbcc58 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.792764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b1bb3-5bd1-456c-8601-09fa71b41805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.793440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b1bb3-5bd1-456c-8601-09fa71b41805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.793721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b1bb3-5bd1-456c-8601-09fa71b41805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.793991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b1bb3-5bd1-456c-8601-09fa71b41805 admin admin] PATH_INFO: `/v3/projects/9d30cd9e49a74d3bacc58dd5a272ea75/users/4410fa6f22f24b40a40d26a8ecd55962/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.809615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b1bb3-5bd1-456c-8601-09fa71b41805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d30cd9e49a74d3bacc58dd5a272ea75, user_id=4410fa6f22f24b40a40d26a8ecd55962, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.812865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b1bb3-5bd1-456c-8601-09fa71b41805 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.826532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1180/2337] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:42:55 2022] PUT /identity/v3/projects/9d30cd9e49a74d3bacc58dd5a272ea75/users/4410fa6f22f24b40a40d26a8ecd55962/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:55.830848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55034095-9188-4724-9bd4-8d5e723aaef0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:55.841588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55034095-9188-4724-9bd4-8d5e723aaef0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:55.967194 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a569f42-3643-4f38-8bba-a09e5cf9cb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:55.968163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a569f42-3643-4f38-8bba-a09e5cf9cb49 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:55.968545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a569f42-3643-4f38-8bba-a09e5cf9cb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:55.968907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a569f42-3643-4f38-8bba-a09e5cf9cb49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:55.969690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a569f42-3643-4f38-8bba-a09e5cf9cb49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:55.971563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a569f42-3643-4f38-8bba-a09e5cf9cb49 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:55.984590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1158/2338] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:42:55.990590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b1bb3-5bd1-456c-8601-09fa71b41805 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:56.004956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b1bb3-5bd1-456c-8601-09fa71b41805 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:56.011082 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:56.013649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:56.014047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:56.014420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] PATH_INFO: `/v3/projects/9d30cd9e49a74d3bacc58dd5a272ea75/users/4410fa6f22f24b40a40d26a8ecd55962/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:56.016570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d30cd9e49a74d3bacc58dd5a272ea75, user_id=4410fa6f22f24b40a40d26a8ecd55962, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:56.019391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:56.034583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1181/2339] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:42:55 2022] PUT /identity/v3/projects/9d30cd9e49a74d3bacc58dd5a272ea75/users/4410fa6f22f24b40a40d26a8ecd55962/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:56.049074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0975e2-c552-4271-8cfa-ee3e7c1ee4cf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:56.049633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0975e2-c552-4271-8cfa-ee3e7c1ee4cf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:56.050035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0975e2-c552-4271-8cfa-ee3e7c1ee4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:56.086291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-af0975e2-c552-4271-8cfa-ee3e7c1ee4cf None None] MFA Rules not processed for user `4410fa6f22f24b40a40d26a8ecd55962`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:56.088687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-af0975e2-c552-4271-8cfa-ee3e7c1ee4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:56.183915 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:56.217949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:56.226004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:56.233944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1159/2340] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:56.237396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0975e2-c552-4271-8cfa-ee3e7c1ee4cf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:56.239623 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-af0975e2-c552-4271-8cfa-ee3e7c1ee4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:56.337655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58c9a0-dc1c-435e-ab0d-c802aa344ba9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:56.340641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0975e2-c552-4271-8cfa-ee3e7c1ee4cf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:56.342644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:56.344272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:56.344576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:56.344843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:56.347520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce851cf0509947e783dbd5b4809bc897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:56.348427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:56.348926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:56.349600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:56.349766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:56.352831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:56.428332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:56.429827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:56.437022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:56.448900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1182/2341] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:56.449813 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:56.451107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:56.455461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:56.457877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:56.471801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1160/2342] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:56.523290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b2a90-6be3-4d62-853b-8e8a91b4ebf5 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:56.527291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659a6d1-472b-46c2-91ee-27e36bb259d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:56.528030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659a6d1-472b-46c2-91ee-27e36bb259d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:56.528408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659a6d1-472b-46c2-91ee-27e36bb259d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:56.528776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659a6d1-472b-46c2-91ee-27e36bb259d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:56.542334 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659a6d1-472b-46c2-91ee-27e36bb259d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:56.544025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659a6d1-472b-46c2-91ee-27e36bb259d3 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:56.557541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1183/2343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:57.163856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:57.241577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160200b-91d5-4b3c-a25e-9c1152d026d7 service glance] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:57.246074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:57.246714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:57.246994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:57.247272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] PATH_INFO: `/v3/users/fc29b2649cb147f0a4ce06d1b5f2da61` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:57.248626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc29b2649cb147f0a4ce06d1b5f2da61)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:57.250413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:57.292028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:57.306479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc29b2649cb147f0a4ce06d1b5f2da61'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:57.315256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc29b2649cb147f0a4ce06d1b5f2da61'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:57.316157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc29b2649cb147f0a4ce06d1b5f2da61'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:57.324429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc29b2649cb147f0a4ce06d1b5f2da61'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:57.345905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc29b2649cb147f0a4ce06d1b5f2da61'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:57.354170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1161/2344] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:42:57 2022] DELETE /identity/v3/users/fc29b2649cb147f0a4ce06d1b5f2da61 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:57.358758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a049fc-90cf-4a68-bf7d-26fa1e680040 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:57.359243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a049fc-90cf-4a68-bf7d-26fa1e680040 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:57.359817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a049fc-90cf-4a68-bf7d-26fa1e680040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:57.381462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-66a049fc-90cf-4a68-bf7d-26fa1e680040 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:57.382787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-66a049fc-90cf-4a68-bf7d-26fa1e680040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:57.453400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1184/2345] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:57.463261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:57.464973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:57.529612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cf76a-ddb3-45d0-9cd7-27c5991fc57b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:57.533597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890b10b-a329-4330-b429-b8cf1966dd70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:57.534223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890b10b-a329-4330-b429-b8cf1966dd70 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:57.534503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890b10b-a329-4330-b429-b8cf1966dd70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:57.534770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890b10b-a329-4330-b429-b8cf1966dd70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:57.544551 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890b10b-a329-4330-b429-b8cf1966dd70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:57.546623 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890b10b-a329-4330-b429-b8cf1966dd70 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:57.557141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1162/2346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:42:57.937469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a049fc-90cf-4a68-bf7d-26fa1e680040 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:57.939981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-66a049fc-90cf-4a68-bf7d-26fa1e680040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:57.955560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a049fc-90cf-4a68-bf7d-26fa1e680040 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:57.964325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:57.965329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:57.966010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:57.966512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] PATH_INFO: `/v3/projects/7fdeef4dbd0c43f1a27216b3a12ee99e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:57.968728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fdeef4dbd0c43f1a27216b3a12ee99e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:57.970818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:58.043174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fdeef4dbd0c43f1a27216b3a12ee99e', 'request_id': 'req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.052439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fdeef4dbd0c43f1a27216b3a12ee99e', 'request_id': 'req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.063281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fdeef4dbd0c43f1a27216b3a12ee99e', 'request_id': 'req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.086714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] The token cache is being invalidate because project 7fdeef4dbd0c43f1a27216b3a12ee99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:42:58.087268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.088727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.094792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1185/2347] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:42:57 2022] DELETE /identity/v3/projects/7fdeef4dbd0c43f1a27216b3a12ee99e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:58.096037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890b10b-a329-4330-b429-b8cf1966dd70 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:58.108160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890b10b-a329-4330-b429-b8cf1966dd70 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:58.184060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:58.184809 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:58.184966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:58.185417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] PATH_INFO: `/v3/users/4410fa6f22f24b40a40d26a8ecd55962` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:58.186579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4410fa6f22f24b40a40d26a8ecd55962)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:58.188494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:58.235317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4410fa6f22f24b40a40d26a8ecd55962'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.242585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4410fa6f22f24b40a40d26a8ecd55962'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.243238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4410fa6f22f24b40a40d26a8ecd55962'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.244579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:58.253721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4410fa6f22f24b40a40d26a8ecd55962'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.269199 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d8a17b88-9c0c-49b5-8d3d-d68f3738a0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4410fa6f22f24b40a40d26a8ecd55962'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.277325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1163/2348] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:42:58 2022] DELETE /identity/v3/users/4410fa6f22f24b40a40d26a8ecd55962 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:58.595105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:58.597535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:58.614676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad0c5-1a87-4c17-b65e-3bd5a9cc9c9f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:58.650137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:58.686996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:58.688017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:58.688513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:58.688991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] PATH_INFO: `/v3/projects/9d30cd9e49a74d3bacc58dd5a272ea75` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:58.690227 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d30cd9e49a74d3bacc58dd5a272ea75)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:58.692251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:58.742679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d30cd9e49a74d3bacc58dd5a272ea75', 'request_id': 'req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.748535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d30cd9e49a74d3bacc58dd5a272ea75', 'request_id': 'req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.755006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d30cd9e49a74d3bacc58dd5a272ea75', 'request_id': 'req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.768996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] The token cache is being invalidate because project 9d30cd9e49a74d3bacc58dd5a272ea75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:42:58.769325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.769986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f0501ae-3fd5-43f7-b274-c9f83e8ee982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:42:58.771504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1186/2349] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:42:58 2022] DELETE /identity/v3/projects/9d30cd9e49a74d3bacc58dd5a272ea75 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:58.777496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c008b-6ef6-4e55-9695-929aac14dd0f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:58.777787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c008b-6ef6-4e55-9695-929aac14dd0f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:58.778059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c008b-6ef6-4e55-9695-929aac14dd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:58.802460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-df9c008b-6ef6-4e55-9695-929aac14dd0f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:58.803916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-df9c008b-6ef6-4e55-9695-929aac14dd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:58.920412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1164/2350] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:42:58.923469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92da5ce-3dcc-4d52-a1c5-cfc7ccbebdeb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:58.923826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92da5ce-3dcc-4d52-a1c5-cfc7ccbebdeb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:58.924161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92da5ce-3dcc-4d52-a1c5-cfc7ccbebdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:58.928595 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d185aa-a644-468c-bb70-485164492cef None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:58.928888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d185aa-a644-468c-bb70-485164492cef None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:58.929164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d185aa-a644-468c-bb70-485164492cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:58.948519 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a92da5ce-3dcc-4d52-a1c5-cfc7ccbebdeb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:58.950247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a92da5ce-3dcc-4d52-a1c5-cfc7ccbebdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:58.965349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1187/2351] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:58.967434 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92da5ce-3dcc-4d52-a1c5-cfc7ccbebdeb None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:58.979796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92da5ce-3dcc-4d52-a1c5-cfc7ccbebdeb None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:58.981041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-41d185aa-a644-468c-bb70-485164492cef None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:58.986600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-41d185aa-a644-468c-bb70-485164492cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:58.989758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:58.991285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:58.991796 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:58.992312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:58.993507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:42:58.994040 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:58.996715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.020006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1188/2352] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:42:59.035283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.053422 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77776b-53d0-49ed-9ea7-ec96af3bdfe5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.060630 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.062059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.062602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.063117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.064312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.067502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.076271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1165/2353] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 10 13:42:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:59.115151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1189/2354] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:42:59 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:42:59.119895 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d185aa-a644-468c-bb70-485164492cef None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.136487 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d185aa-a644-468c-bb70-485164492cef None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.142276 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.145177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:59.239593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855caf2-b8af-4e4b-bab9-c98e45ba752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.241066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855caf2-b8af-4e4b-bab9-c98e45ba752e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.241896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6e3bd-33f3-49fb-84c6-3d5ded7eca8e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.242773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855caf2-b8af-4e4b-bab9-c98e45ba752e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.243377 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855caf2-b8af-4e4b-bab9-c98e45ba752e admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.245147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855caf2-b8af-4e4b-bab9-c98e45ba752e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.247975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855caf2-b8af-4e4b-bab9-c98e45ba752e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.253386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce047f-c1c0-4e8d-a18f-a907c5b85394 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.254254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce047f-c1c0-4e8d-a18f-a907c5b85394 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.254761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce047f-c1c0-4e8d-a18f-a907c5b85394 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.255177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce047f-c1c0-4e8d-a18f-a907c5b85394 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.276170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:59.294752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1166/2355] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:42:59 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:42:59.299294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855caf2-b8af-4e4b-bab9-c98e45ba752e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.312351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855caf2-b8af-4e4b-bab9-c98e45ba752e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.318781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613faaee-ff85-4a90-a131-8a92a8765b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.319617 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613faaee-ff85-4a90-a131-8a92a8765b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.319888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613faaee-ff85-4a90-a131-8a92a8765b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.320160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613faaee-ff85-4a90-a131-8a92a8765b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.321532 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613faaee-ff85-4a90-a131-8a92a8765b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.323864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613faaee-ff85-4a90-a131-8a92a8765b9f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.325517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:59.328666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce047f-c1c0-4e8d-a18f-a907c5b85394 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.330797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce047f-c1c0-4e8d-a18f-a907c5b85394 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.340624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1190/2356] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:42:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:42:59.345850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1167/2357] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:42:59.361258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce047f-c1c0-4e8d-a18f-a907c5b85394 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.376149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce047f-c1c0-4e8d-a18f-a907c5b85394 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.392710 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3ed08-828c-4e77-8eab-331ab965e926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.393387 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3ed08-828c-4e77-8eab-331ab965e926 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.393669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3ed08-828c-4e77-8eab-331ab965e926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.393942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3ed08-828c-4e77-8eab-331ab965e926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.394592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3ed08-828c-4e77-8eab-331ab965e926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.396462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3ed08-828c-4e77-8eab-331ab965e926 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.407993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1191/2358] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:42:59.412198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613faaee-ff85-4a90-a131-8a92a8765b9f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.423411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613faaee-ff85-4a90-a131-8a92a8765b9f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.428724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6150e-2eb7-4856-929f-b826e9586301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.429468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6150e-2eb7-4856-929f-b826e9586301 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.429687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6150e-2eb7-4856-929f-b826e9586301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.429958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6150e-2eb7-4856-929f-b826e9586301 admin admin] PATH_INFO: `/v3/projects/cb9ebc86c02e4ec4828cc24be82571c7/users/6b6e30b6479c4b95a4b903a21df36611/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.446841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6150e-2eb7-4856-929f-b826e9586301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9ebc86c02e4ec4828cc24be82571c7, user_id=6b6e30b6479c4b95a4b903a21df36611, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.449739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6150e-2eb7-4856-929f-b826e9586301 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.465645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1168/2359] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:42:59 2022] PUT /identity/v3/projects/cb9ebc86c02e4ec4828cc24be82571c7/users/6b6e30b6479c4b95a4b903a21df36611/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:59.476083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eab60-2ab3-4227-a4bc-cbb3becedde3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.476364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eab60-2ab3-4227-a4bc-cbb3becedde3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.477153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eab60-2ab3-4227-a4bc-cbb3becedde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.517058 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-140eab60-2ab3-4227-a4bc-cbb3becedde3 None None] MFA Rules not processed for user `6b6e30b6479c4b95a4b903a21df36611`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:42:59.518638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-140eab60-2ab3-4227-a4bc-cbb3becedde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:42:59.600593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1192/2360] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:42:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:42:59.605351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6150e-2eb7-4856-929f-b826e9586301 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.673939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6150e-2eb7-4856-929f-b826e9586301 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.678755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9447cfd-5ca0-40f5-bd81-9a0821bd925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.679368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9447cfd-5ca0-40f5-bd81-9a0821bd925c admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.679642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9447cfd-5ca0-40f5-bd81-9a0821bd925c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.679909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9447cfd-5ca0-40f5-bd81-9a0821bd925c admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.680578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9447cfd-5ca0-40f5-bd81-9a0821bd925c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.682495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9447cfd-5ca0-40f5-bd81-9a0821bd925c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.713291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1169/2361] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:42:59 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:42:59.715020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140eab60-2ab3-4227-a4bc-cbb3becedde3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.727730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140eab60-2ab3-4227-a4bc-cbb3becedde3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.791168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2a1a-2c46-4169-84d4-bb1e3ce2864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.791693 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2a1a-2c46-4169-84d4-bb1e3ce2864c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.791970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2a1a-2c46-4169-84d4-bb1e3ce2864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.792240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2a1a-2c46-4169-84d4-bb1e3ce2864c admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.792904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d2a1a-2c46-4169-84d4-bb1e3ce2864c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.794877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d2a1a-2c46-4169-84d4-bb1e3ce2864c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.816838 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1193/2362] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:42:59 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:42:59.820786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9447cfd-5ca0-40f5-bd81-9a0821bd925c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.831689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9447cfd-5ca0-40f5-bd81-9a0821bd925c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.836383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08a12b-11bc-42b9-97af-47656bee3d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.836989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a12b-11bc-42b9-97af-47656bee3d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.837294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a12b-11bc-42b9-97af-47656bee3d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.837547 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a12b-11bc-42b9-97af-47656bee3d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.838189 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08a12b-11bc-42b9-97af-47656bee3d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.839767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08a12b-11bc-42b9-97af-47656bee3d6b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.854673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1170/2363] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:42:59.856085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2a1a-2c46-4169-84d4-bb1e3ce2864c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.866420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2a1a-2c46-4169-84d4-bb1e3ce2864c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.870989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5341266-5ab9-41ed-91f4-5a9592aa365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.871635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341266-5ab9-41ed-91f4-5a9592aa365e admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.871908 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341266-5ab9-41ed-91f4-5a9592aa365e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.872175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341266-5ab9-41ed-91f4-5a9592aa365e admin admin] PATH_INFO: `/v3/projects/7cb043d81b5a49e4bc07828f8496df08/users/64addee8351742efad3dce5f9fc668c8/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.888879 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5341266-5ab9-41ed-91f4-5a9592aa365e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb043d81b5a49e4bc07828f8496df08, user_id=64addee8351742efad3dce5f9fc668c8, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.892387 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5341266-5ab9-41ed-91f4-5a9592aa365e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.902008 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:42:59.909502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1194/2364] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:42:59 2022] PUT /identity/v3/projects/7cb043d81b5a49e4bc07828f8496df08/users/64addee8351742efad3dce5f9fc668c8/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:42:59.910855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08a12b-11bc-42b9-97af-47656bee3d6b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:42:59.921155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08a12b-11bc-42b9-97af-47656bee3d6b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:42:59.979339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0fa27-09c8-48a1-8d00-e24f5fb1635a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:42:59.979985 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0fa27-09c8-48a1-8d00-e24f5fb1635a admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:42:59.980291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0fa27-09c8-48a1-8d00-e24f5fb1635a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:42:59.980606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0fa27-09c8-48a1-8d00-e24f5fb1635a admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:42:59.981322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0fa27-09c8-48a1-8d00-e24f5fb1635a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:42:59.982909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0fa27-09c8-48a1-8d00-e24f5fb1635a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:42:59.994543 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1171/2365] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:42:59.998695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5341266-5ab9-41ed-91f4-5a9592aa365e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:00.009698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5341266-5ab9-41ed-91f4-5a9592aa365e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:00.015085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:00.015737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:00.016015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:00.016306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] PATH_INFO: `/v3/projects/7cb043d81b5a49e4bc07828f8496df08/users/64addee8351742efad3dce5f9fc668c8/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:00.018224 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb043d81b5a49e4bc07828f8496df08, user_id=64addee8351742efad3dce5f9fc668c8, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:00.020851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:00.031952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1195/2366] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:42:59 2022] PUT /identity/v3/projects/7cb043d81b5a49e4bc07828f8496df08/users/64addee8351742efad3dce5f9fc668c8/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:43:00.044230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b2fe1-4ec0-491d-ae7a-f8bced620342 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:00.044437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b2fe1-4ec0-491d-ae7a-f8bced620342 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:00.044706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b2fe1-4ec0-491d-ae7a-f8bced620342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:00.074371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4a5b2fe1-4ec0-491d-ae7a-f8bced620342 None None] MFA Rules not processed for user `64addee8351742efad3dce5f9fc668c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:00.075612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4a5b2fe1-4ec0-491d-ae7a-f8bced620342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:00.105358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:00.147221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1172/2367] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:43:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:00.158271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:00.160138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:00.224509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a3784-e2f6-45e2-b120-d0db99d31bda admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:00.228301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56253238-ce8a-43e6-9651-33959766672d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:00.229010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56253238-ce8a-43e6-9651-33959766672d service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:00.229311 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56253238-ce8a-43e6-9651-33959766672d service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:00.229584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56253238-ce8a-43e6-9651-33959766672d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:00.239143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56253238-ce8a-43e6-9651-33959766672d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:00.240580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56253238-ce8a-43e6-9651-33959766672d service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:00.250368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1196/2368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:43:00.993055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b2fe1-4ec0-491d-ae7a-f8bced620342 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:01.087933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b2fe1-4ec0-491d-ae7a-f8bced620342 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:01.093585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:01.094563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:01.094845 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:01.095175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] PATH_INFO: `/v3/users/6b6e30b6479c4b95a4b903a21df36611` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:01.097057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6e30b6479c4b95a4b903a21df36611)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:01.099341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:01.191283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6e30b6479c4b95a4b903a21df36611'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:01.205282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6e30b6479c4b95a4b903a21df36611'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:01.206160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6e30b6479c4b95a4b903a21df36611'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:01.217365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6e30b6479c4b95a4b903a21df36611'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:01.230391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6e30b6479c4b95a4b903a21df36611'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:01.237438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1173/2369] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:43:00 2022] DELETE /identity/v3/users/6b6e30b6479c4b95a4b903a21df36611 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:43:01.242500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bba40-a647-486d-ad32-575f287cc35f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:01.243025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bba40-a647-486d-ad32-575f287cc35f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:01.243142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bba40-a647-486d-ad32-575f287cc35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:01.263665 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-908bba40-a647-486d-ad32-575f287cc35f None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:01.264948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-908bba40-a647-486d-ad32-575f287cc35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:01.335945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1197/2370] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:01.348172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:01.349716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:01.420019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:01.429501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914add0-5d20-4478-96d5-9401d096cb09 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:01.435525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42190aca-0c41-4354-8e71-a709c47b7757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:01.436521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42190aca-0c41-4354-8e71-a709c47b7757 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:01.436867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42190aca-0c41-4354-8e71-a709c47b7757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:01.437150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42190aca-0c41-4354-8e71-a709c47b7757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:01.450247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42190aca-0c41-4354-8e71-a709c47b7757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:01.451881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42190aca-0c41-4354-8e71-a709c47b7757 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:01.463609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1174/2371] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:43:02.132024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bba40-a647-486d-ad32-575f287cc35f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:02.133585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-908bba40-a647-486d-ad32-575f287cc35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:02.152099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bba40-a647-486d-ad32-575f287cc35f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:02.164354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:02.166281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:02.167336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:02.167778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] PATH_INFO: `/v3/projects/cb9ebc86c02e4ec4828cc24be82571c7` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:02.169950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9ebc86c02e4ec4828cc24be82571c7)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:02.173465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:02.244259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9ebc86c02e4ec4828cc24be82571c7', 'request_id': 'req-05550c2c-e55e-4503-99bc-427f13638683'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.251236 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9ebc86c02e4ec4828cc24be82571c7', 'request_id': 'req-05550c2c-e55e-4503-99bc-427f13638683'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.262209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9ebc86c02e4ec4828cc24be82571c7', 'request_id': 'req-05550c2c-e55e-4503-99bc-427f13638683'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.280442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] The token cache is being invalidate because project cb9ebc86c02e4ec4828cc24be82571c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:43:02.280985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.281861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.285314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1198/2372] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:43:02 2022] DELETE /identity/v3/projects/cb9ebc86c02e4ec4828cc24be82571c7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:43:02.287574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42190aca-0c41-4354-8e71-a709c47b7757 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:02.298162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42190aca-0c41-4354-8e71-a709c47b7757 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:02.370724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:02.371528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:02.371879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:02.372241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] PATH_INFO: `/v3/users/64addee8351742efad3dce5f9fc668c8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:02.373605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=64addee8351742efad3dce5f9fc668c8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:02.376318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:02.444522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64addee8351742efad3dce5f9fc668c8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.454159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64addee8351742efad3dce5f9fc668c8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.455587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64addee8351742efad3dce5f9fc668c8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.466928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64addee8351742efad3dce5f9fc668c8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.482003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-36d936de-1ab0-4649-8be6-96672bbb7b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64addee8351742efad3dce5f9fc668c8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:02.490347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1175/2373] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:43:02 2022] DELETE /identity/v3/users/64addee8351742efad3dce5f9fc668c8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:43:02.920402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:02.923406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:02.944350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05550c2c-e55e-4503-99bc-427f13638683 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:02.957100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:03.011645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:03.012378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.012728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.013059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] PATH_INFO: `/v3/projects/7cb043d81b5a49e4bc07828f8496df08` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.014353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb043d81b5a49e4bc07828f8496df08)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:03.016938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:03.069171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb043d81b5a49e4bc07828f8496df08', 'request_id': 'req-9cb314e5-f76b-40d1-835a-3fbac62650da'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:03.074645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb043d81b5a49e4bc07828f8496df08', 'request_id': 'req-9cb314e5-f76b-40d1-835a-3fbac62650da'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:03.082237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb043d81b5a49e4bc07828f8496df08', 'request_id': 'req-9cb314e5-f76b-40d1-835a-3fbac62650da'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:03.096085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] The token cache is being invalidate because project 7cb043d81b5a49e4bc07828f8496df08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:43:03.096323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:03.096962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9cb314e5-f76b-40d1-835a-3fbac62650da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:03.098534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1199/2374] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:43:02 2022] DELETE /identity/v3/projects/7cb043d81b5a49e4bc07828f8496df08 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:43:03.103361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d75a10-4746-4403-ba9b-0e99eb30068e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.103652 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d75a10-4746-4403-ba9b-0e99eb30068e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.104032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d75a10-4746-4403-ba9b-0e99eb30068e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.124219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-17d75a10-4746-4403-ba9b-0e99eb30068e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:03.125459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-17d75a10-4746-4403-ba9b-0e99eb30068e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:03.229156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1176/2375] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:43:03.237585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e144c-a2a9-4cec-b9ec-c94fed0325e4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.238259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e144c-a2a9-4cec-b9ec-c94fed0325e4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.238653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e144c-a2a9-4cec-b9ec-c94fed0325e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.277037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-bf2e144c-a2a9-4cec-b9ec-c94fed0325e4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:03.278595 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bf2e144c-a2a9-4cec-b9ec-c94fed0325e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:03.295396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1200/2376] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:03.300259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d75a10-4746-4403-ba9b-0e99eb30068e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:03.311866 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d75a10-4746-4403-ba9b-0e99eb30068e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:03.316849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:03.317814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.318392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.319005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.319847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:43:03.320355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:03.322172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:03.325268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:03.338576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1177/2377] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:43:03.343122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e144c-a2a9-4cec-b9ec-c94fed0325e4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:03.361213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e144c-a2a9-4cec-b9ec-c94fed0325e4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:03.372647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48cd0-24fe-4be6-899c-45ac8b00b5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:03.373781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48cd0-24fe-4be6-899c-45ac8b00b5ee admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.374237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48cd0-24fe-4be6-899c-45ac8b00b5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.374735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48cd0-24fe-4be6-899c-45ac8b00b5ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.376567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade48cd0-24fe-4be6-899c-45ac8b00b5ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:03.379690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade48cd0-24fe-4be6-899c-45ac8b00b5ee admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:03.413477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1201/2378] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:43:03 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:43:03.417711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:03.428757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55903691-efee-4599-bfce-c61a234e9e77 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:03.509383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79523a6-dc64-4ac6-8244-62db7508eaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:03.510306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79523a6-dc64-4ac6-8244-62db7508eaba admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.510431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79523a6-dc64-4ac6-8244-62db7508eaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.510692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79523a6-dc64-4ac6-8244-62db7508eaba admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.511394 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79523a6-dc64-4ac6-8244-62db7508eaba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:03.513515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79523a6-dc64-4ac6-8244-62db7508eaba admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:03.545876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1178/2379] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:43:03 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:43:03.551994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48cd0-24fe-4be6-899c-45ac8b00b5ee admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:03.573130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48cd0-24fe-4be6-899c-45ac8b00b5ee admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:03.581510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea4d7-47ba-46f4-bb33-6bebd657aac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:03.582987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea4d7-47ba-46f4-bb33-6bebd657aac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.583432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea4d7-47ba-46f4-bb33-6bebd657aac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.583848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea4d7-47ba-46f4-bb33-6bebd657aac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.584678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea4d7-47ba-46f4-bb33-6bebd657aac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:03.586839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea4d7-47ba-46f4-bb33-6bebd657aac1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:03.607044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1202/2380] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:43:03.608336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79523a6-dc64-4ac6-8244-62db7508eaba admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:03.621535 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79523a6-dc64-4ac6-8244-62db7508eaba admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:03.629995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba42f2-9f44-4716-a843-3688658bab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:03.631078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba42f2-9f44-4716-a843-3688658bab7d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.631457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba42f2-9f44-4716-a843-3688658bab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.632086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba42f2-9f44-4716-a843-3688658bab7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.633167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba42f2-9f44-4716-a843-3688658bab7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:03.635298 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba42f2-9f44-4716-a843-3688658bab7d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:03.653142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1179/2381] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:43:03.657112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea4d7-47ba-46f4-bb33-6bebd657aac1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:03.672078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea4d7-47ba-46f4-bb33-6bebd657aac1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:03.676184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:03.677330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.677609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.677876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] PATH_INFO: `/v3/projects/6cafc9dd5a5341c7a823844c010eb15a/users/ed0e66db98634e028d8706ce14488309/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.699954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cafc9dd5a5341c7a823844c010eb15a, user_id=ed0e66db98634e028d8706ce14488309, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:03.703373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:03.725050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1203/2382] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:43:03 2022] PUT /identity/v3/projects/6cafc9dd5a5341c7a823844c010eb15a/users/ed0e66db98634e028d8706ce14488309/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:43:03.729933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c501-12cb-44ee-bf90-3f049e5ddc83 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.730405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c501-12cb-44ee-bf90-3f049e5ddc83 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.730749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c501-12cb-44ee-bf90-3f049e5ddc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.759333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5364c501-12cb-44ee-bf90-3f049e5ddc83 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:03.761380 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5364c501-12cb-44ee-bf90-3f049e5ddc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:03.858310 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1180/2383] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:03.881302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:03.883587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:03.959491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:03.977286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5b15f-db32-460c-8784-4999c995bf16 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:03.982794 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:03.984069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:03.984443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:03.984812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:03.996156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:03.997636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:04.009899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1204/2384] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:43:04.920352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a03fb4-a001-4250-bf00-454d20169432 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:04.920940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a03fb4-a001-4250-bf00-454d20169432 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:04.921053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a03fb4-a001-4250-bf00-454d20169432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:04.953386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e5a03fb4-a001-4250-bf00-454d20169432 None None] MFA Rules not processed for user `ed0e66db98634e028d8706ce14488309`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:04.954907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e5a03fb4-a001-4250-bf00-454d20169432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:05.001056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:05.003023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:05.022583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c851-be9e-49b1-8372-a0718248e98f service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:05.037448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e6ff8-7a35-462c-8e03-f7fc3fe51a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:05.038471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6ff8-7a35-462c-8e03-f7fc3fe51a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:05.038842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6ff8-7a35-462c-8e03-f7fc3fe51a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:05.039501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6ff8-7a35-462c-8e03-f7fc3fe51a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:05.041647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-366e6ff8-7a35-462c-8e03-f7fc3fe51a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:05.056932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e6ff8-7a35-462c-8e03-f7fc3fe51a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:05.059800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e6ff8-7a35-462c-8e03-f7fc3fe51a6f service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:05.063463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1181/2385] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:05.080909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a03fb4-a001-4250-bf00-454d20169432 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:05.084746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e5a03fb4-a001-4250-bf00-454d20169432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:05.086827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1205/2386] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:43:05.092227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:05.157326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a03fb4-a001-4250-bf00-454d20169432 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:05.160940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:05.161593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:05.161862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:05.162588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:05.171758 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:05.173191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:05.183068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1182/2387] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:43:06.633188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cedacd-a3f8-4fc7-8071-44e0f20905f3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:06.636259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cedacd-a3f8-4fc7-8071-44e0f20905f3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:06.636698 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cedacd-a3f8-4fc7-8071-44e0f20905f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:06.671151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-06cedacd-a3f8-4fc7-8071-44e0f20905f3 None None] MFA Rules not processed for user `ed0e66db98634e028d8706ce14488309`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:06.673453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-06cedacd-a3f8-4fc7-8071-44e0f20905f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:06.689526 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1206/2388] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:43:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:07.779392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:07.782136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:07.913697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467646f0-f7f1-43d3-9f08-6159e0aea2db service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:07.920009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:07.920852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:07.921147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:07.921451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:07.923647 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:07.997488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:07.999206 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:08.020476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1183/2389] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:43:10.024940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cedacd-a3f8-4fc7-8071-44e0f20905f3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:10.026326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-06cedacd-a3f8-4fc7-8071-44e0f20905f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:10.069473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cedacd-a3f8-4fc7-8071-44e0f20905f3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:10.075486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:10.077304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:10.078057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:10.078659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] PATH_INFO: `/v3/users/ccc10397cf6a4b9084bcd6cdb12b8607` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:10.080244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc10397cf6a4b9084bcd6cdb12b8607)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:10.083438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:10.162177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc10397cf6a4b9084bcd6cdb12b8607'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.176627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc10397cf6a4b9084bcd6cdb12b8607'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.177186 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc10397cf6a4b9084bcd6cdb12b8607'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.195834 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc10397cf6a4b9084bcd6cdb12b8607'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.196606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc10397cf6a4b9084bcd6cdb12b8607'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.212200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1207/2390] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:43:10 2022] DELETE /identity/v3/users/ccc10397cf6a4b9084bcd6cdb12b8607 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:43:10.534328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:10.593393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cf81c-38cb-44dc-9df4-1c1532ed7464 service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:10.696265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:10.733546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:10.735464 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:10.736071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:10.736682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] PATH_INFO: `/v3/projects/2158cd74bdb44c47a905e34e97c5807f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:10.739467 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2158cd74bdb44c47a905e34e97c5807f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:10.744739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:10.842288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2158cd74bdb44c47a905e34e97c5807f', 'request_id': 'req-b94fbb1c-adac-4bde-9003-fec2733b767d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.860811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2158cd74bdb44c47a905e34e97c5807f', 'request_id': 'req-b94fbb1c-adac-4bde-9003-fec2733b767d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.874230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2158cd74bdb44c47a905e34e97c5807f', 'request_id': 'req-b94fbb1c-adac-4bde-9003-fec2733b767d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.894229 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] The token cache is being invalidate because project 2158cd74bdb44c47a905e34e97c5807f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:43:10.894441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.895148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:10.896788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1184/2391] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:43:10 2022] DELETE /identity/v3/projects/2158cd74bdb44c47a905e34e97c5807f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:43:10.900691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:10.902637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:10.917719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5dfa6-aaef-41e6-a3cc-97d9f8eecd29 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:11.009592 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:11.011343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:11.011343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:11.011343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] PATH_INFO: `/v3/users/4cf79e1933aa4ec2898aa0dec9dae6bc` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:11.012766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf79e1933aa4ec2898aa0dec9dae6bc)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:11.015309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:11.058783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:11.069127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf79e1933aa4ec2898aa0dec9dae6bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.078612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf79e1933aa4ec2898aa0dec9dae6bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.079316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf79e1933aa4ec2898aa0dec9dae6bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.095777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf79e1933aa4ec2898aa0dec9dae6bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.096537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf79e1933aa4ec2898aa0dec9dae6bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.117355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1208/2392] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:43:10 2022] DELETE /identity/v3/users/4cf79e1933aa4ec2898aa0dec9dae6bc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:43:11.712529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:11.718546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:11.739947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fbb1c-adac-4bde-9003-fec2733b767d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:11.840788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:11.841605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:11.841982 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:11.843058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] PATH_INFO: `/v3/projects/c28df7f6904d4d5e9f1100742b99d6eb` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:11.845732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28df7f6904d4d5e9f1100742b99d6eb)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:11.848853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:11.944769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28df7f6904d4d5e9f1100742b99d6eb', 'request_id': 'req-422a815b-435e-436e-bde1-541207690b6b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.950595 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28df7f6904d4d5e9f1100742b99d6eb', 'request_id': 'req-422a815b-435e-436e-bde1-541207690b6b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.957767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28df7f6904d4d5e9f1100742b99d6eb', 'request_id': 'req-422a815b-435e-436e-bde1-541207690b6b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.972559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] The token cache is being invalidate because project c28df7f6904d4d5e9f1100742b99d6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:43:11.972839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.975489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-422a815b-435e-436e-bde1-541207690b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:43:11.977056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1185/2393] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:43:11 2022] DELETE /identity/v3/projects/c28df7f6904d4d5e9f1100742b99d6eb => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:43:21.110708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=15,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:21.969428 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:22.623130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:22.625358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:22.736412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb9373-0871-4fa2-aa40-505b0aca660f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:22.819266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec465d-3ecc-4a86-ae00-b69f760c5d5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:22.819944 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec465d-3ecc-4a86-ae00-b69f760c5d5d service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:22.820237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec465d-3ecc-4a86-ae00-b69f760c5d5d service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:22.820499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec465d-3ecc-4a86-ae00-b69f760c5d5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:22.822480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a6ec465d-3ecc-4a86-ae00-b69f760c5d5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:22.839566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec465d-3ecc-4a86-ae00-b69f760c5d5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:22.841547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec465d-3ecc-4a86-ae00-b69f760c5d5d service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:22.855455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1209/2394] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:43:23.996814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3676-a190-4cf9-b381-6cdef8b67367 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:23.997726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3676-a190-4cf9-b381-6cdef8b67367 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:23.998976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3676-a190-4cf9-b381-6cdef8b67367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:24.019746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-307c3676-a190-4cf9-b381-6cdef8b67367 None None] MFA Rules not processed for user `ed0e66db98634e028d8706ce14488309`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:24.021264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-307c3676-a190-4cf9-b381-6cdef8b67367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:24.078044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1186/2395] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:43:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:25.302443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035082b8-f0e8-490e-8922-ca67f35067fb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:25.303208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035082b8-f0e8-490e-8922-ca67f35067fb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:25.303208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035082b8-f0e8-490e-8922-ca67f35067fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:25.327969 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-035082b8-f0e8-490e-8922-ca67f35067fb None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:25.329578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-035082b8-f0e8-490e-8922-ca67f35067fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:25.405271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1210/2396] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:43:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:43:25.419804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3676-a190-4cf9-b381-6cdef8b67367 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:25.422229 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-307c3676-a190-4cf9-b381-6cdef8b67367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:25.498943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3676-a190-4cf9-b381-6cdef8b67367 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:25.503905 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ac416-4696-4bad-af21-f20387b857dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:25.504522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac416-4696-4bad-af21-f20387b857dd service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:25.504792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac416-4696-4bad-af21-f20387b857dd service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:25.505055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac416-4696-4bad-af21-f20387b857dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:25.513843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ac416-4696-4bad-af21-f20387b857dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:25.515420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ac416-4696-4bad-af21-f20387b857dd service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:25.523892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1187/2397] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:43:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:43:26.733045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035082b8-f0e8-490e-8922-ca67f35067fb None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:26.735583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-035082b8-f0e8-490e-8922-ca67f35067fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:26.753931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035082b8-f0e8-490e-8922-ca67f35067fb None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:26.761072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:26.762034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:26.762509 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:26.762922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:26.765756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:26.783851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:26.786161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:26.826086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1211/2398] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:43:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:43:35.520336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=57 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:36.803752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:45.696686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6f021-203e-4a09-a439-a28d664dd9cb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:45.697170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6f021-203e-4a09-a439-a28d664dd9cb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:45.697361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6f021-203e-4a09-a439-a28d664dd9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:45.717066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-93b6f021-203e-4a09-a439-a28d664dd9cb None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:45.718207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-93b6f021-203e-4a09-a439-a28d664dd9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:45.725372 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1188/2399] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:45.737488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:45.749345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee40d22-3cb4-48c3-a560-ef2b361c495f service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:45.754321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96238c12-df6c-46cb-a8b0-9d011778b15f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:45.754941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96238c12-df6c-46cb-a8b0-9d011778b15f service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:45.755211 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96238c12-df6c-46cb-a8b0-9d011778b15f service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:45.755476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96238c12-df6c-46cb-a8b0-9d011778b15f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:45.764531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96238c12-df6c-46cb-a8b0-9d011778b15f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:45.766129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96238c12-df6c-46cb-a8b0-9d011778b15f service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:45.774367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1212/2400] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:43:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:43:46.024637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9ad39-0f11-4b4c-919f-32698a636f5f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:46.025153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9ad39-0f11-4b4c-919f-32698a636f5f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:46.025607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9ad39-0f11-4b4c-919f-32698a636f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:46.057737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f2a9ad39-0f11-4b4c-919f-32698a636f5f None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:43:46.059877 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f2a9ad39-0f11-4b4c-919f-32698a636f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:43:46.072031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1189/2401] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 10 13:43:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:43:46.095407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96238c12-df6c-46cb-a8b0-9d011778b15f service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:43:46.111342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96238c12-df6c-46cb-a8b0-9d011778b15f service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:43:46.117676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2902350-ad58-4499-b12e-98408f667177 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:43:46.118345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2902350-ad58-4499-b12e-98408f667177 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:43:46.118746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2902350-ad58-4499-b12e-98408f667177 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:43:46.119018 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2902350-ad58-4499-b12e-98408f667177 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:43:46.129160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2902350-ad58-4499-b12e-98408f667177 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:43:46.130971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2902350-ad58-4499-b12e-98408f667177 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:43:46.143456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1213/2402] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:43:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:43:56.049255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:43:56.139249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=12 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:02.389517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3601b-daa8-42bd-89a8-33a41998c8f1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:02.390037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3601b-daa8-42bd-89a8-33a41998c8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:02.390037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3601b-daa8-42bd-89a8-33a41998c8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:02.412004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c5a3601b-daa8-42bd-89a8-33a41998c8f1 None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:02.414256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c5a3601b-daa8-42bd-89a8-33a41998c8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:02.423842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1190/2403] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 10 13:44:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:02.437094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2902350-ad58-4499-b12e-98408f667177 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:02.447662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2902350-ad58-4499-b12e-98408f667177 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:02.453775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065c283-0f8f-46c9-90cd-55d7e682fa9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:02.454382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065c283-0f8f-46c9-90cd-55d7e682fa9b service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:02.454662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065c283-0f8f-46c9-90cd-55d7e682fa9b service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:02.454935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065c283-0f8f-46c9-90cd-55d7e682fa9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:02.463354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065c283-0f8f-46c9-90cd-55d7e682fa9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:02.465360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065c283-0f8f-46c9-90cd-55d7e682fa9b service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:02.475044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1214/2404] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:44:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:44:03.120313 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15c5cd-57ef-43dd-b30a-a4e9e8956312 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:03.120596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15c5cd-57ef-43dd-b30a-a4e9e8956312 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:03.120868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15c5cd-57ef-43dd-b30a-a4e9e8956312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:03.140761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1f15c5cd-57ef-43dd-b30a-a4e9e8956312 None None] MFA Rules not processed for user `1e0811556a0542b685c98dabbaed5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:03.141997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1f15c5cd-57ef-43dd-b30a-a4e9e8956312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:03.149117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1191/2405] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 10 13:44:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:03.161162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065c283-0f8f-46c9-90cd-55d7e682fa9b service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:03.172579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065c283-0f8f-46c9-90cd-55d7e682fa9b service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:03.177628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10209fea1bcc43509534b7e20f635c75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:03.178218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:03.178495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:03.178763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:03.187302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:03.188883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:03.196780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1215/2406] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 10 13:44:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:44:04.372282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15c5cd-57ef-43dd-b30a-a4e9e8956312 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:04.373723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1f15c5cd-57ef-43dd-b30a-a4e9e8956312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:04.427637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:04.429138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:04.468672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15c5cd-57ef-43dd-b30a-a4e9e8956312 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:04.473616 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:04.474299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:04.474641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:04.474994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:04.476810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:04.488680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:04.490405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:04.497830 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8303b0-4328-4409-8981-243c393dd248 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:04.501541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1192/2407] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:04.505708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:04.506446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:04.506822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:04.507190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] PATH_INFO: `/v3/users/ed0e66db98634e028d8706ce14488309` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:04.508377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0e66db98634e028d8706ce14488309)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:04.510402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:04.560511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0e66db98634e028d8706ce14488309'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.569396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0e66db98634e028d8706ce14488309'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.570077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0e66db98634e028d8706ce14488309'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.582157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0e66db98634e028d8706ce14488309'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.582871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5386abfb-e087-4a6b-9c26-59e4fbb9078c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0e66db98634e028d8706ce14488309'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.595546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1216/2408] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:04 2022] DELETE /identity/v3/users/ed0e66db98634e028d8706ce14488309 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:04.787779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:04.811760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fb7fa-a743-4f2b-84f8-e2df8e613c63 service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:04.888576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:04.889325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:04.889666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:04.890002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] PATH_INFO: `/v3/projects/6cafc9dd5a5341c7a823844c010eb15a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:04.891304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cafc9dd5a5341c7a823844c010eb15a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:04.893970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:04.953153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cafc9dd5a5341c7a823844c010eb15a', 'request_id': 'req-9c58be19-ff83-4034-87ef-15b69ffa475e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.960390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cafc9dd5a5341c7a823844c010eb15a', 'request_id': 'req-9c58be19-ff83-4034-87ef-15b69ffa475e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.969050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cafc9dd5a5341c7a823844c010eb15a', 'request_id': 'req-9c58be19-ff83-4034-87ef-15b69ffa475e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.988160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] The token cache is being invalidate because project 6cafc9dd5a5341c7a823844c010eb15a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:04.988942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.989646 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9c58be19-ff83-4034-87ef-15b69ffa475e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:04.994515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1193/2409] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:04 2022] DELETE /identity/v3/projects/6cafc9dd5a5341c7a823844c010eb15a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:04.996125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81c11d-528a-41a7-8185-166c2197c5f2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:04.996477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81c11d-528a-41a7-8185-166c2197c5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:04.996904 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81c11d-528a-41a7-8185-166c2197c5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.034841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9a81c11d-528a-41a7-8185-166c2197c5f2 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:05.036112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9a81c11d-528a-41a7-8185-166c2197c5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:05.126130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:05.147358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1217/2410] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:05.155124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc99c0-d242-4b28-88ca-36ceb75dcb1e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.155403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc99c0-d242-4b28-88ca-36ceb75dcb1e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.155666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc99c0-d242-4b28-88ca-36ceb75dcb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.170934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:05.176944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-88cc99c0-d242-4b28-88ca-36ceb75dcb1e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:05.179098 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-88cc99c0-d242-4b28-88ca-36ceb75dcb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:05.187321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1194/2411] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:05.191438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81c11d-528a-41a7-8185-166c2197c5f2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:05.201260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81c11d-528a-41a7-8185-166c2197c5f2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:05.205769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:05.206348 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.206619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.206883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.207517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:05.207805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:05.209531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:05.221377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1218/2412] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:44:05.225525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc99c0-d242-4b28-88ca-36ceb75dcb1e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:05.235232 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc99c0-d242-4b28-88ca-36ceb75dcb1e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:05.240067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49c144-cc6b-43eb-b05a-5a9b71c776fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:05.240655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49c144-cc6b-43eb-b05a-5a9b71c776fb admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.240929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49c144-cc6b-43eb-b05a-5a9b71c776fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.241198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49c144-cc6b-43eb-b05a-5a9b71c776fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.241838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49c144-cc6b-43eb-b05a-5a9b71c776fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:05.243723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49c144-cc6b-43eb-b05a-5a9b71c776fb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:05.270077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1195/2413] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:05 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:05.274180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:05.284497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff10231-a36d-4a6a-8520-d3a063c3c31a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:05.338172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f8cae-896b-4a37-b692-c597c6c7dd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:05.338942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f8cae-896b-4a37-b692-c597c6c7dd7c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.339091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f8cae-896b-4a37-b692-c597c6c7dd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.339356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f8cae-896b-4a37-b692-c597c6c7dd7c admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.339977 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f8cae-896b-4a37-b692-c597c6c7dd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:05.342548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f8cae-896b-4a37-b692-c597c6c7dd7c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:05.364994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1219/2414] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:05 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:05.368356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49c144-cc6b-43eb-b05a-5a9b71c776fb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:05.378381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49c144-cc6b-43eb-b05a-5a9b71c776fb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:05.383311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f3e0f-3be5-48a1-acf2-bc9fabe69d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:05.383896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3e0f-3be5-48a1-acf2-bc9fabe69d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.384167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3e0f-3be5-48a1-acf2-bc9fabe69d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.384457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3e0f-3be5-48a1-acf2-bc9fabe69d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.385080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f3e0f-3be5-48a1-acf2-bc9fabe69d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:05.386708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f3e0f-3be5-48a1-acf2-bc9fabe69d78 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:05.396658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1196/2415] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:05.400579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f8cae-896b-4a37-b692-c597c6c7dd7c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:05.411049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f8cae-896b-4a37-b692-c597c6c7dd7c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:05.415621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d48376-ad9b-4711-b863-33c193ae657e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:05.416218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d48376-ad9b-4711-b863-33c193ae657e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.416495 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d48376-ad9b-4711-b863-33c193ae657e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.416784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d48376-ad9b-4711-b863-33c193ae657e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.417450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d48376-ad9b-4711-b863-33c193ae657e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:05.419769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d48376-ad9b-4711-b863-33c193ae657e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:05.431458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1220/2416] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:05.435608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f3e0f-3be5-48a1-acf2-bc9fabe69d78 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:05.445849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f3e0f-3be5-48a1-acf2-bc9fabe69d78 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:05.451276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:05.451947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.452286 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.452541 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] PATH_INFO: `/v3/projects/ddbd8e8e12fe45cdb52bfec6117e6ae4/users/3d45203a8bc6455ea4fdba21dcaf7f18/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.469941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbd8e8e12fe45cdb52bfec6117e6ae4, user_id=3d45203a8bc6455ea4fdba21dcaf7f18, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:05.472700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:05.486095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1197/2417] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:05 2022] PUT /identity/v3/projects/ddbd8e8e12fe45cdb52bfec6117e6ae4/users/3d45203a8bc6455ea4fdba21dcaf7f18/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:05.496564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ac8b3-a4ae-4122-b09c-f62e12c81631 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.496858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ac8b3-a4ae-4122-b09c-f62e12c81631 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.497160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ac8b3-a4ae-4122-b09c-f62e12c81631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.528851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-681ac8b3-a4ae-4122-b09c-f62e12c81631 None None] MFA Rules not processed for user `3d45203a8bc6455ea4fdba21dcaf7f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:05.530094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-681ac8b3-a4ae-4122-b09c-f62e12c81631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:05.595369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1221/2418] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:05.606663 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:05.608335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:05.667029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92d4fb-ead1-4e01-9f5f-377a6228e508 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:05.671831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8a8ff-0e11-4d1f-8ad0-f676bb5bc842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:05.672855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8a8ff-0e11-4d1f-8ad0-f676bb5bc842 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:05.673384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8a8ff-0e11-4d1f-8ad0-f676bb5bc842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:05.673733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8a8ff-0e11-4d1f-8ad0-f676bb5bc842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:05.682533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8a8ff-0e11-4d1f-8ad0-f676bb5bc842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:05.684027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8a8ff-0e11-4d1f-8ad0-f676bb5bc842 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:05.694819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1198/2419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:08.256036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ac8b3-a4ae-4122-b09c-f62e12c81631 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:08.306726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:08.322899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ac8b3-a4ae-4122-b09c-f62e12c81631 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:08.328046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:08.328669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:08.328946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:08.329216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] PATH_INFO: `/v3/users/3d45203a8bc6455ea4fdba21dcaf7f18` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:08.330356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d45203a8bc6455ea4fdba21dcaf7f18)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:08.332309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:08.394983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d45203a8bc6455ea4fdba21dcaf7f18'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:08.402767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d45203a8bc6455ea4fdba21dcaf7f18'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:08.403895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d45203a8bc6455ea4fdba21dcaf7f18'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:08.416159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d45203a8bc6455ea4fdba21dcaf7f18'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:08.416801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d45203a8bc6455ea4fdba21dcaf7f18'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:08.429889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1222/2420] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:08 2022] DELETE /identity/v3/users/3d45203a8bc6455ea4fdba21dcaf7f18 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:08.434351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ca44-8cb5-48a9-bb8b-7a347dfad28c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:08.434833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ca44-8cb5-48a9-bb8b-7a347dfad28c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:08.434949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ca44-8cb5-48a9-bb8b-7a347dfad28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:08.455050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-42e0ca44-8cb5-48a9-bb8b-7a347dfad28c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:08.456322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-42e0ca44-8cb5-48a9-bb8b-7a347dfad28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:08.518081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1199/2421] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:08.528663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:08.530165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:08.588001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d021c7-7e50-4b06-82bb-f7bcbd7acacf admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:08.592077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947cbe1-1d3f-4549-8d9e-59b6a04b1e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:08.592798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947cbe1-1d3f-4549-8d9e-59b6a04b1e11 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:08.593083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947cbe1-1d3f-4549-8d9e-59b6a04b1e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:08.593398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947cbe1-1d3f-4549-8d9e-59b6a04b1e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:08.604116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947cbe1-1d3f-4549-8d9e-59b6a04b1e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:08.607089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947cbe1-1d3f-4549-8d9e-59b6a04b1e11 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:08.618119 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1223/2422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:09.001176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ca44-8cb5-48a9-bb8b-7a347dfad28c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:09.002709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-42e0ca44-8cb5-48a9-bb8b-7a347dfad28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:09.016388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ca44-8cb5-48a9-bb8b-7a347dfad28c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:09.022157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:09.022860 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.023164 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.023431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] PATH_INFO: `/v3/projects/ddbd8e8e12fe45cdb52bfec6117e6ae4` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.024574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbd8e8e12fe45cdb52bfec6117e6ae4)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:09.026501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:09.074319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:09.082918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbd8e8e12fe45cdb52bfec6117e6ae4', 'request_id': 'req-d4451c4a-8a81-4e12-b808-f20dd1ede80c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:09.089772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbd8e8e12fe45cdb52bfec6117e6ae4', 'request_id': 'req-d4451c4a-8a81-4e12-b808-f20dd1ede80c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:09.096434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbd8e8e12fe45cdb52bfec6117e6ae4', 'request_id': 'req-d4451c4a-8a81-4e12-b808-f20dd1ede80c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:09.111105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] The token cache is being invalidate because project ddbd8e8e12fe45cdb52bfec6117e6ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:09.111475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:09.112120 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d4451c4a-8a81-4e12-b808-f20dd1ede80c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:09.115441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1200/2423] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:09 2022] DELETE /identity/v3/projects/ddbd8e8e12fe45cdb52bfec6117e6ae4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:09.119963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20962c2-b8dd-4341-81b9-155a3e75289c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.120244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20962c2-b8dd-4341-81b9-155a3e75289c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.120515 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20962c2-b8dd-4341-81b9-155a3e75289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.141531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b20962c2-b8dd-4341-81b9-155a3e75289c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:09.142780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b20962c2-b8dd-4341-81b9-155a3e75289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:09.229744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1224/2424] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:09.232757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7dcae-1b15-4d01-b8ed-554f6e648a77 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.233042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7dcae-1b15-4d01-b8ed-554f6e648a77 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.233343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7dcae-1b15-4d01-b8ed-554f6e648a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.254522 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-30c7dcae-1b15-4d01-b8ed-554f6e648a77 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:09.255621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-30c7dcae-1b15-4d01-b8ed-554f6e648a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:09.264317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1201/2425] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:09.268559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20962c2-b8dd-4341-81b9-155a3e75289c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:09.279755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20962c2-b8dd-4341-81b9-155a3e75289c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:09.285337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:09.286014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.286362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.286692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.287400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:09.287762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:09.289440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:09.296692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:09.303594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1225/2426] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:44:09.307710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7dcae-1b15-4d01-b8ed-554f6e648a77 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:09.317470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7dcae-1b15-4d01-b8ed-554f6e648a77 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:09.323832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f999f-fcb9-4907-b31b-886ddce33d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:09.324016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f999f-fcb9-4907-b31b-886ddce33d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.324235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f999f-fcb9-4907-b31b-886ddce33d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.324497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f999f-fcb9-4907-b31b-886ddce33d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.325689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f999f-fcb9-4907-b31b-886ddce33d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:09.327628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f999f-fcb9-4907-b31b-886ddce33d16 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:09.357066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1202/2427] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:09 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:09.361190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:09.371677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e51dd-4103-4cec-9975-0077ce8d0bcd admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:09.429670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86aaf7-9535-4d54-8ba7-06d3f05804f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:09.430593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86aaf7-9535-4d54-8ba7-06d3f05804f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.430866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86aaf7-9535-4d54-8ba7-06d3f05804f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.431134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86aaf7-9535-4d54-8ba7-06d3f05804f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.431756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86aaf7-9535-4d54-8ba7-06d3f05804f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:09.433719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86aaf7-9535-4d54-8ba7-06d3f05804f0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:09.459514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1226/2428] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:09 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:09.464390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f999f-fcb9-4907-b31b-886ddce33d16 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:09.477458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f999f-fcb9-4907-b31b-886ddce33d16 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:09.482155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad264f04-a40c-43c1-8429-a1d0fb6f6d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:09.482762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad264f04-a40c-43c1-8429-a1d0fb6f6d0e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.483031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad264f04-a40c-43c1-8429-a1d0fb6f6d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.483292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad264f04-a40c-43c1-8429-a1d0fb6f6d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.483981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad264f04-a40c-43c1-8429-a1d0fb6f6d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:09.485711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad264f04-a40c-43c1-8429-a1d0fb6f6d0e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:09.504349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1203/2429] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:09.505860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86aaf7-9535-4d54-8ba7-06d3f05804f0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:09.516703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86aaf7-9535-4d54-8ba7-06d3f05804f0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:09.530921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a8216-a066-476e-b8bf-11d3d9edad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:09.531534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a8216-a066-476e-b8bf-11d3d9edad3b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.531812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a8216-a066-476e-b8bf-11d3d9edad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.532084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a8216-a066-476e-b8bf-11d3d9edad3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.532727 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a8216-a066-476e-b8bf-11d3d9edad3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:09.534360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a8216-a066-476e-b8bf-11d3d9edad3b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:09.545536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1227/2430] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:09.551193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad264f04-a40c-43c1-8429-a1d0fb6f6d0e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:09.562143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad264f04-a40c-43c1-8429-a1d0fb6f6d0e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:09.566831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:09.567502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.567777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.568052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] PATH_INFO: `/v3/projects/e353e7c1c8b64117bb7a7c3a92d0d4b8/users/0a854a8c58714ae7b60639cd25b30220/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.585184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e353e7c1c8b64117bb7a7c3a92d0d4b8, user_id=0a854a8c58714ae7b60639cd25b30220, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:09.588000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:09.603187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1204/2431] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:09 2022] PUT /identity/v3/projects/e353e7c1c8b64117bb7a7c3a92d0d4b8/users/0a854a8c58714ae7b60639cd25b30220/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:09.613898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76dca4-257a-4df5-8e5b-c3cb46e82871 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.614174 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76dca4-257a-4df5-8e5b-c3cb46e82871 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.614439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76dca4-257a-4df5-8e5b-c3cb46e82871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.643772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7d76dca4-257a-4df5-8e5b-c3cb46e82871 None None] MFA Rules not processed for user `0a854a8c58714ae7b60639cd25b30220`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:09.645049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7d76dca4-257a-4df5-8e5b-c3cb46e82871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:09.711065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1228/2432] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:09.723314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:09.725040 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:09.788515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e4675-f09c-45e8-90de-87bd9236ffef admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:09.792144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:09.792765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:09.793059 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:09.793314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:09.804893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:09.806403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:09.818150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1205/2433] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:12.502828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76dca4-257a-4df5-8e5b-c3cb46e82871 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:12.504398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7d76dca4-257a-4df5-8e5b-c3cb46e82871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:12.516776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76dca4-257a-4df5-8e5b-c3cb46e82871 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:12.520788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:12.521426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:12.521706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:12.521977 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:12.523722 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:12.584056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:12.585517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:12.595262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1229/2434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:16.399882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:16.402169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:16.497062 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a431d9-ee64-4bf3-87f2-ffc2fc06f45d service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:16.502364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:16.503363 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:16.503738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:16.504048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:16.506977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:16.538728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:16.608291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:16.613191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:16.634631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1206/2435] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:44:20.009029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:20.010758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:20.023702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:20.030455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419c320-bad7-4dbd-b779-6092f3d8fdf1 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:20.036661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:20.037475 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.037787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.038056 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] PATH_INFO: `/v3/users/a3d9e5dc719c430eac6b5c8d8b814566` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.039749 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3d9e5dc719c430eac6b5c8d8b814566)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:20.041768 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:20.099378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3d9e5dc719c430eac6b5c8d8b814566'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.108015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3d9e5dc719c430eac6b5c8d8b814566'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.108978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3d9e5dc719c430eac6b5c8d8b814566'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.121666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3d9e5dc719c430eac6b5c8d8b814566'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.122471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bd30e7fa-4e71-4999-a384-eaea4137d0a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3d9e5dc719c430eac6b5c8d8b814566'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.135281 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1230/2436] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:20 2022] DELETE /identity/v3/users/a3d9e5dc719c430eac6b5c8d8b814566 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:20.317726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:20.330897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca7584-a574-421f-9096-2fff43af4f2c service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:20.390188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:20.390762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.391037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.391305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] PATH_INFO: `/v3/projects/48e5556cbbfd4b00aa4f6055cd6238b2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.392413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48e5556cbbfd4b00aa4f6055cd6238b2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:20.394348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:20.441468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48e5556cbbfd4b00aa4f6055cd6238b2', 'request_id': 'req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.447624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48e5556cbbfd4b00aa4f6055cd6238b2', 'request_id': 'req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.453737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48e5556cbbfd4b00aa4f6055cd6238b2', 'request_id': 'req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.468674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] The token cache is being invalidate because project 48e5556cbbfd4b00aa4f6055cd6238b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:20.468959 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.469604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac92a6a3-bbb1-4053-bd0b-9b95822a0120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:20.471083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1207/2437] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:20 2022] DELETE /identity/v3/projects/48e5556cbbfd4b00aa4f6055cd6238b2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:20.477994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7224e9-ea59-4290-bb40-ba321754e7d1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.477994 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7224e9-ea59-4290-bb40-ba321754e7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.478402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7224e9-ea59-4290-bb40-ba321754e7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.506735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0d7224e9-ea59-4290-bb40-ba321754e7d1 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:20.508332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0d7224e9-ea59-4290-bb40-ba321754e7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:20.604543 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1231/2438] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:20.615956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cf06e-ff64-40bb-b5ff-f9293d8a1e91 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.616538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cf06e-ff64-40bb-b5ff-f9293d8a1e91 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.617319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cf06e-ff64-40bb-b5ff-f9293d8a1e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.654101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-335cf06e-ff64-40bb-b5ff-f9293d8a1e91 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:20.656462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-335cf06e-ff64-40bb-b5ff-f9293d8a1e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:20.669708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1208/2439] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:20.674274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7224e9-ea59-4290-bb40-ba321754e7d1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:20.685835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33a77-1782-48ec-8cdb-76103cc321db None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.686117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33a77-1782-48ec-8cdb-76103cc321db None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.686403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33a77-1782-48ec-8cdb-76103cc321db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.689356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7224e9-ea59-4290-bb40-ba321754e7d1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:20.695361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:20.695971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.696247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.696516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.697163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:20.697515 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:20.699117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:20.708890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b7f33a77-1782-48ec-8cdb-76103cc321db None None] MFA Rules not processed for user `411ff3f0306848ee91108343982806ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:20.710515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b7f33a77-1782-48ec-8cdb-76103cc321db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:20.712537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1232/2440] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:44:20.716746 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:20.728052 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce248d8-0dc5-4988-9354-ff4f3bdcf7eb admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:20.734105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ad80c-acb8-4090-a0b3-e2d29430c0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:20.734876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ad80c-acb8-4090-a0b3-e2d29430c0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.735218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ad80c-acb8-4090-a0b3-e2d29430c0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.735218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ad80c-acb8-4090-a0b3-e2d29430c0b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.735831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ad80c-acb8-4090-a0b3-e2d29430c0b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:20.738374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ad80c-acb8-4090-a0b3-e2d29430c0b1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:20.752112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:20.769201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1233/2441] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:20 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:20.772031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1209/2442] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:20.773777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ad80c-acb8-4090-a0b3-e2d29430c0b1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:20.821083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:20.845993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ad80c-acb8-4090-a0b3-e2d29430c0b1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:20.853890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf38efc-0d16-4007-bc0e-4a7daedf36db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:20.854537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf38efc-0d16-4007-bc0e-4a7daedf36db admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.854831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf38efc-0d16-4007-bc0e-4a7daedf36db admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.855235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf38efc-0d16-4007-bc0e-4a7daedf36db admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.856437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf38efc-0d16-4007-bc0e-4a7daedf36db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:20.858516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf38efc-0d16-4007-bc0e-4a7daedf36db admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:20.888063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1234/2443] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:20 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:20.892376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33a77-1782-48ec-8cdb-76103cc321db None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:20.908619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33a77-1782-48ec-8cdb-76103cc321db None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:20.914187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dcb47-12d4-47d2-82cf-5cf14ed1ea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:20.914625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dcb47-12d4-47d2-82cf-5cf14ed1ea45 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.915072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dcb47-12d4-47d2-82cf-5cf14ed1ea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.917025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dcb47-12d4-47d2-82cf-5cf14ed1ea45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.917559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443dcb47-12d4-47d2-82cf-5cf14ed1ea45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:20.919794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443dcb47-12d4-47d2-82cf-5cf14ed1ea45 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:20.937089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1210/2444] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:20.938319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf38efc-0d16-4007-bc0e-4a7daedf36db admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:20.949287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf38efc-0d16-4007-bc0e-4a7daedf36db admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:20.955675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c633dd2-4297-44e2-bef1-1c80804ed840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:20.956323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c633dd2-4297-44e2-bef1-1c80804ed840 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:20.956601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c633dd2-4297-44e2-bef1-1c80804ed840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:20.956869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c633dd2-4297-44e2-bef1-1c80804ed840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:20.957755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c633dd2-4297-44e2-bef1-1c80804ed840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:20.959798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c633dd2-4297-44e2-bef1-1c80804ed840 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:20.991866 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dcb47-12d4-47d2-82cf-5cf14ed1ea45 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:20.992907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1235/2445] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:21.005902 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dcb47-12d4-47d2-82cf-5cf14ed1ea45 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:21.011737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:21.013140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:21.013456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:21.013728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] PATH_INFO: `/v3/projects/c590b4c64c1746d88db3a5584db53a97/users/de09286315614eedaec2899d05a6e4d4/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:21.030099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c590b4c64c1746d88db3a5584db53a97, user_id=de09286315614eedaec2899d05a6e4d4, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:21.032917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:21.047764 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1211/2446] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:20 2022] PUT /identity/v3/projects/c590b4c64c1746d88db3a5584db53a97/users/de09286315614eedaec2899d05a6e4d4/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:21.058101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959854-9123-4e60-94db-0ef6aa033cbd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:21.059309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959854-9123-4e60-94db-0ef6aa033cbd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:21.059309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959854-9123-4e60-94db-0ef6aa033cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:21.092078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3e959854-9123-4e60-94db-0ef6aa033cbd None None] MFA Rules not processed for user `de09286315614eedaec2899d05a6e4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:21.093315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3e959854-9123-4e60-94db-0ef6aa033cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:21.173828 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1236/2447] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:21.178365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:21.180521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:21.184917 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e959854-9123-4e60-94db-0ef6aa033cbd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:21.186619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3e959854-9123-4e60-94db-0ef6aa033cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:21.288111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c08de-6b3b-49bf-a811-780154898177 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:21.289417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e959854-9123-4e60-94db-0ef6aa033cbd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:21.297526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776309ed-dd14-4d01-91bb-8b5538c9968c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:21.298174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776309ed-dd14-4d01-91bb-8b5538c9968c service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:21.299123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c67056-4775-4d30-bfbd-c5ace2f28f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:21.299787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c67056-4775-4d30-bfbd-c5ace2f28f9d service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:21.300009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c67056-4775-4d30-bfbd-c5ace2f28f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:21.300339 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c67056-4775-4d30-bfbd-c5ace2f28f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:21.301095 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776309ed-dd14-4d01-91bb-8b5538c9968c service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:21.301429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776309ed-dd14-4d01-91bb-8b5538c9968c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:21.304913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-776309ed-dd14-4d01-91bb-8b5538c9968c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:21.321669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c67056-4775-4d30-bfbd-c5ace2f28f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:21.323318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c67056-4775-4d30-bfbd-c5ace2f28f9d service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:21.327886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776309ed-dd14-4d01-91bb-8b5538c9968c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:21.330183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776309ed-dd14-4d01-91bb-8b5538c9968c service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:21.335148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1237/2448] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:44:21.340706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1212/2449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:44:31.329781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:31.336538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=47,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:33.351562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c67056-4775-4d30-bfbd-c5ace2f28f9d service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:33.426147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c67056-4775-4d30-bfbd-c5ace2f28f9d service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:33.432007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:33.432705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:33.433418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:33.433418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] PATH_INFO: `/v3/users/de09286315614eedaec2899d05a6e4d4` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:33.434442 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de09286315614eedaec2899d05a6e4d4)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:33.436357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:33.508234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de09286315614eedaec2899d05a6e4d4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:33.525754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de09286315614eedaec2899d05a6e4d4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:33.526435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de09286315614eedaec2899d05a6e4d4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:33.544375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de09286315614eedaec2899d05a6e4d4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:33.545372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de09286315614eedaec2899d05a6e4d4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:33.563153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1238/2450] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:33 2022] DELETE /identity/v3/users/de09286315614eedaec2899d05a6e4d4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:33.571531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93496c5-3736-44bb-bb02-ea0303450aa5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:33.572484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93496c5-3736-44bb-bb02-ea0303450aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:33.572755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93496c5-3736-44bb-bb02-ea0303450aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:33.594582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a93496c5-3736-44bb-bb02-ea0303450aa5 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:33.596056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a93496c5-3736-44bb-bb02-ea0303450aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:33.674460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1213/2451] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:33.684278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:33.685933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:33.815016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174486ac-6270-440f-9502-7f94a6c1d556 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:33.820599 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a251ed-721f-4169-a81c-a63906ede75f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:33.821349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a251ed-721f-4169-a81c-a63906ede75f service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:33.821769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a251ed-721f-4169-a81c-a63906ede75f service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:33.823451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a251ed-721f-4169-a81c-a63906ede75f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:33.868358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a251ed-721f-4169-a81c-a63906ede75f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:33.869865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a251ed-721f-4169-a81c-a63906ede75f service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:33.896320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1239/2452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:34.255350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93496c5-3736-44bb-bb02-ea0303450aa5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:34.256938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a93496c5-3736-44bb-bb02-ea0303450aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:34.283175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93496c5-3736-44bb-bb02-ea0303450aa5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:34.290539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:34.292192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.292525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.292849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] PATH_INFO: `/v3/projects/c590b4c64c1746d88db3a5584db53a97` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.295322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c590b4c64c1746d88db3a5584db53a97)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:34.297374 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:34.364989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c590b4c64c1746d88db3a5584db53a97', 'request_id': 'req-e07f0363-aedf-4d02-a731-bb6178d605fe'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:34.375148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c590b4c64c1746d88db3a5584db53a97', 'request_id': 'req-e07f0363-aedf-4d02-a731-bb6178d605fe'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:34.388480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c590b4c64c1746d88db3a5584db53a97', 'request_id': 'req-e07f0363-aedf-4d02-a731-bb6178d605fe'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:34.407099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] The token cache is being invalidate because project c590b4c64c1746d88db3a5584db53a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:34.407612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:34.408368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e07f0363-aedf-4d02-a731-bb6178d605fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:34.410105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1214/2453] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:34 2022] DELETE /identity/v3/projects/c590b4c64c1746d88db3a5584db53a97 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:34.415642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903819-e9f3-445c-8afc-0c4ce0be5bab None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.416194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903819-e9f3-445c-8afc-0c4ce0be5bab None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.416194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903819-e9f3-445c-8afc-0c4ce0be5bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.441474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e7903819-e9f3-445c-8afc-0c4ce0be5bab None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:34.442618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e7903819-e9f3-445c-8afc-0c4ce0be5bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:34.464480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:34.531380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1240/2454] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:34.545121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526af00-dc54-4b6d-af3f-f9ce088113ca None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.545810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526af00-dc54-4b6d-af3f-f9ce088113ca None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.546054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526af00-dc54-4b6d-af3f-f9ce088113ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.586205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3526af00-dc54-4b6d-af3f-f9ce088113ca None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:34.587526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3526af00-dc54-4b6d-af3f-f9ce088113ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:34.598198 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1215/2455] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:34.604751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7903819-e9f3-445c-8afc-0c4ce0be5bab None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:34.629074 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7903819-e9f3-445c-8afc-0c4ce0be5bab None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:34.636417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:34.637459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.637922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.638373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.639423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:34.639914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:34.642852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:34.662240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1241/2456] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:44:34.669167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526af00-dc54-4b6d-af3f-f9ce088113ca None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:34.679823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526af00-dc54-4b6d-af3f-f9ce088113ca None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:34.687009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b728ae8-2809-4754-98d2-89b9fee8cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:34.687629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728ae8-2809-4754-98d2-89b9fee8cc09 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.687907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728ae8-2809-4754-98d2-89b9fee8cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.688177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728ae8-2809-4754-98d2-89b9fee8cc09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.688797 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b728ae8-2809-4754-98d2-89b9fee8cc09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:34.690725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b728ae8-2809-4754-98d2-89b9fee8cc09 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:34.720961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1216/2457] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:34 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:34.725728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:34.736258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1fa5d-37f0-4ff1-8eaf-d77cc45ab99b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:34.802586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86490b57-ef6e-41d6-92df-e128f1389d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:34.803203 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86490b57-ef6e-41d6-92df-e128f1389d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.803480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86490b57-ef6e-41d6-92df-e128f1389d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.803750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86490b57-ef6e-41d6-92df-e128f1389d4e admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.804374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86490b57-ef6e-41d6-92df-e128f1389d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:34.806381 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86490b57-ef6e-41d6-92df-e128f1389d4e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:34.833578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1242/2458] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:34 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:34.834336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b728ae8-2809-4754-98d2-89b9fee8cc09 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:34.846203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b728ae8-2809-4754-98d2-89b9fee8cc09 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:34.850739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473d29-bcdc-4beb-b27c-12a4b017befa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:34.851337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473d29-bcdc-4beb-b27c-12a4b017befa admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.851609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473d29-bcdc-4beb-b27c-12a4b017befa admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.851875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473d29-bcdc-4beb-b27c-12a4b017befa admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.852506 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe473d29-bcdc-4beb-b27c-12a4b017befa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:34.854129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe473d29-bcdc-4beb-b27c-12a4b017befa admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:34.864321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1217/2459] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:34.868469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86490b57-ef6e-41d6-92df-e128f1389d4e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:34.878660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86490b57-ef6e-41d6-92df-e128f1389d4e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:34.883321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50797f82-f786-49cc-89c8-a38c474e5759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:34.883941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50797f82-f786-49cc-89c8-a38c474e5759 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.884361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50797f82-f786-49cc-89c8-a38c474e5759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.884634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50797f82-f786-49cc-89c8-a38c474e5759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.885308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50797f82-f786-49cc-89c8-a38c474e5759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:34.886903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50797f82-f786-49cc-89c8-a38c474e5759 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:34.900605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1243/2460] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:34.911637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473d29-bcdc-4beb-b27c-12a4b017befa admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:34.923160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473d29-bcdc-4beb-b27c-12a4b017befa admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:34.928813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b643-5f1b-4b88-b115-4e7d4034d071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:34.929766 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b643-5f1b-4b88-b115-4e7d4034d071 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.930042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b643-5f1b-4b88-b115-4e7d4034d071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.930307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b643-5f1b-4b88-b115-4e7d4034d071 admin admin] PATH_INFO: `/v3/projects/b57e802cbad543f887fc582e9e3d7159/users/ff46d01136c944b4a428c6ce9e213b65/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:34.947315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903b643-5f1b-4b88-b115-4e7d4034d071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57e802cbad543f887fc582e9e3d7159, user_id=ff46d01136c944b4a428c6ce9e213b65, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:34.950599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903b643-5f1b-4b88-b115-4e7d4034d071 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:34.970473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1218/2461] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:34 2022] PUT /identity/v3/projects/b57e802cbad543f887fc582e9e3d7159/users/ff46d01136c944b4a428c6ce9e213b65/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:34.984024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9e773-f591-44f3-a348-70b7d575b0d5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:34.984574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9e773-f591-44f3-a348-70b7d575b0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:34.985181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9e773-f591-44f3-a348-70b7d575b0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.036881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-90f9e773-f591-44f3-a348-70b7d575b0d5 None None] MFA Rules not processed for user `ff46d01136c944b4a428c6ce9e213b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:35.040134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-90f9e773-f591-44f3-a348-70b7d575b0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:35.124940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1244/2462] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:35.131839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b643-5f1b-4b88-b115-4e7d4034d071 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:35.155346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:35.206652 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b643-5f1b-4b88-b115-4e7d4034d071 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:35.211437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e418c3-3b95-4622-94db-1782a04204c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:35.212717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e418c3-3b95-4622-94db-1782a04204c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.212717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e418c3-3b95-4622-94db-1782a04204c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.212717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e418c3-3b95-4622-94db-1782a04204c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.213277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e418c3-3b95-4622-94db-1782a04204c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:35.215231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e418c3-3b95-4622-94db-1782a04204c2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:35.252311 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1219/2463] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:35 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:35.253985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9e773-f591-44f3-a348-70b7d575b0d5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:35.265618 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9e773-f591-44f3-a348-70b7d575b0d5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:35.279826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:35.339935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3908a9d-ff3c-4b74-b891-be30ce620f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:35.340455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3908a9d-ff3c-4b74-b891-be30ce620f9b admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.340728 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3908a9d-ff3c-4b74-b891-be30ce620f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.340992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3908a9d-ff3c-4b74-b891-be30ce620f9b admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.341659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3908a9d-ff3c-4b74-b891-be30ce620f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:35.343597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3908a9d-ff3c-4b74-b891-be30ce620f9b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:35.365452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1245/2464] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:35 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:35.369448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e418c3-3b95-4622-94db-1782a04204c2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:35.381362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e418c3-3b95-4622-94db-1782a04204c2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:35.389361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1cf7c-0cc7-4d06-b195-20b2287c2ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:35.390070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1cf7c-0cc7-4d06-b195-20b2287c2ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.390383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1cf7c-0cc7-4d06-b195-20b2287c2ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.390724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1cf7c-0cc7-4d06-b195-20b2287c2ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.392080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1cf7c-0cc7-4d06-b195-20b2287c2ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:35.393791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1cf7c-0cc7-4d06-b195-20b2287c2ba1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:35.406324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1220/2465] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:35.410628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3908a9d-ff3c-4b74-b891-be30ce620f9b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:35.421911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3908a9d-ff3c-4b74-b891-be30ce620f9b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:35.427088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a624aa-f024-47e9-ae81-44276f76d271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:35.427827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a624aa-f024-47e9-ae81-44276f76d271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.428029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a624aa-f024-47e9-ae81-44276f76d271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.428305 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a624aa-f024-47e9-ae81-44276f76d271 admin admin] PATH_INFO: `/v3/projects/86d0b9cf1e8a4fd69dda845935a5ad27/users/8a8136e4709d4e71823ad0e92f31633b/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.446344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a624aa-f024-47e9-ae81-44276f76d271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d0b9cf1e8a4fd69dda845935a5ad27, user_id=8a8136e4709d4e71823ad0e92f31633b, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:35.448999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a624aa-f024-47e9-ae81-44276f76d271 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:35.473416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1246/2466] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:35 2022] PUT /identity/v3/projects/86d0b9cf1e8a4fd69dda845935a5ad27/users/8a8136e4709d4e71823ad0e92f31633b/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:35.478909 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1cf7c-0cc7-4d06-b195-20b2287c2ba1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:35.489858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1cf7c-0cc7-4d06-b195-20b2287c2ba1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:35.547314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ac7ae-944c-43ca-bcd1-a4048ba16013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:35.547924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ac7ae-944c-43ca-bcd1-a4048ba16013 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.548197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ac7ae-944c-43ca-bcd1-a4048ba16013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.548462 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ac7ae-944c-43ca-bcd1-a4048ba16013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.549144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ac7ae-944c-43ca-bcd1-a4048ba16013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:35.550742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ac7ae-944c-43ca-bcd1-a4048ba16013 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:35.564180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1221/2467] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:35.565450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a624aa-f024-47e9-ae81-44276f76d271 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:35.576954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a624aa-f024-47e9-ae81-44276f76d271 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:35.582051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:35.582780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.583121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.583467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] PATH_INFO: `/v3/projects/86d0b9cf1e8a4fd69dda845935a5ad27/users/8a8136e4709d4e71823ad0e92f31633b/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.585490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d0b9cf1e8a4fd69dda845935a5ad27, user_id=8a8136e4709d4e71823ad0e92f31633b, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:35.588116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:35.599563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1247/2468] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:35 2022] PUT /identity/v3/projects/86d0b9cf1e8a4fd69dda845935a5ad27/users/8a8136e4709d4e71823ad0e92f31633b/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:35.611540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638aa26-a591-4242-978f-90f9f0b9cee7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.612026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638aa26-a591-4242-978f-90f9f0b9cee7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.612304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638aa26-a591-4242-978f-90f9f0b9cee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.644263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8638aa26-a591-4242-978f-90f9f0b9cee7 None None] MFA Rules not processed for user `8a8136e4709d4e71823ad0e92f31633b`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:35.645718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8638aa26-a591-4242-978f-90f9f0b9cee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:35.729827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1222/2469] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:35.743479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:35.745287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:35.822707 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181eb70-b64d-4c9d-8b7e-e9b0237cd9d8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:35.828147 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:35.828789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.829262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.829444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.843546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:35.845172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:35.855059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1248/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:44:35.940965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638aa26-a591-4242-978f-90f9f0b9cee7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:35.950689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638aa26-a591-4242-978f-90f9f0b9cee7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:35.954261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b7ef2-d633-4722-a90e-3690014bb1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:35.954867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b7ef2-d633-4722-a90e-3690014bb1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:35.955185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b7ef2-d633-4722-a90e-3690014bb1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:35.955458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b7ef2-d633-4722-a90e-3690014bb1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:35.969622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:36.015921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b7ef2-d633-4722-a90e-3690014bb1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:36.017389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b7ef2-d633-4722-a90e-3690014bb1c2 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:36.026134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1223/2471] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:38.288681 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:38.290206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:38.346611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55588b2-6d67-4cb8-872f-0a14cee60865 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:38.350496 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb92858-b6d1-4198-a5d1-fae197f44385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:38.351084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb92858-b6d1-4198-a5d1-fae197f44385 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:38.351363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb92858-b6d1-4198-a5d1-fae197f44385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:38.351626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb92858-b6d1-4198-a5d1-fae197f44385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:38.353360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6bb92858-b6d1-4198-a5d1-fae197f44385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:38.384708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:38.424264 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb92858-b6d1-4198-a5d1-fae197f44385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:38.425772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb92858-b6d1-4198-a5d1-fae197f44385 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:38.436597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1249/2472] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:38.892644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b7ef2-d633-4722-a90e-3690014bb1c2 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:38.911818 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b7ef2-d633-4722-a90e-3690014bb1c2 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:38.916562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:38.917248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:38.917524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:38.917789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] PATH_INFO: `/v3/users/ff46d01136c944b4a428c6ce9e213b65` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:38.918922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff46d01136c944b4a428c6ce9e213b65)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:38.920835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:38.970215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff46d01136c944b4a428c6ce9e213b65'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:38.983261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff46d01136c944b4a428c6ce9e213b65'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:38.983873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff46d01136c944b4a428c6ce9e213b65'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:38.992839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff46d01136c944b4a428c6ce9e213b65'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:39.005258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff46d01136c944b4a428c6ce9e213b65'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:39.011465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1224/2473] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:38 2022] DELETE /identity/v3/users/ff46d01136c944b4a428c6ce9e213b65 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:39.016380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3890b2-3c7e-4462-a4c3-4bf8aa044621 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:39.016660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3890b2-3c7e-4462-a4c3-4bf8aa044621 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:39.016929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3890b2-3c7e-4462-a4c3-4bf8aa044621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:39.035970 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3a3890b2-3c7e-4462-a4c3-4bf8aa044621 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:39.037165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3a3890b2-3c7e-4462-a4c3-4bf8aa044621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:39.107851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1250/2474] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:39.118534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:39.120116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:39.201396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1344008-14be-4445-bd46-9175e836b509 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:39.205140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc82b9-28fe-4cc2-8a46-e11ac9401b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:39.205777 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc82b9-28fe-4cc2-8a46-e11ac9401b5a service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:39.206056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc82b9-28fe-4cc2-8a46-e11ac9401b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:39.206315 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc82b9-28fe-4cc2-8a46-e11ac9401b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:39.216181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc82b9-28fe-4cc2-8a46-e11ac9401b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:39.217675 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc82b9-28fe-4cc2-8a46-e11ac9401b5a service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:39.227424 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1225/2475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:39.702109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3890b2-3c7e-4462-a4c3-4bf8aa044621 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:39.703643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3a3890b2-3c7e-4462-a4c3-4bf8aa044621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:39.719232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3890b2-3c7e-4462-a4c3-4bf8aa044621 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:39.725646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:39.726729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:39.727088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:39.727468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] PATH_INFO: `/v3/projects/b57e802cbad543f887fc582e9e3d7159` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:39.728763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57e802cbad543f887fc582e9e3d7159)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:39.730916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:39.798113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57e802cbad543f887fc582e9e3d7159', 'request_id': 'req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:39.804249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57e802cbad543f887fc582e9e3d7159', 'request_id': 'req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:39.811956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57e802cbad543f887fc582e9e3d7159', 'request_id': 'req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:39.830022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] The token cache is being invalidate because project b57e802cbad543f887fc582e9e3d7159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:39.830309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:39.830985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:39.832889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1251/2476] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:39 2022] DELETE /identity/v3/projects/b57e802cbad543f887fc582e9e3d7159 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:39.837361 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc82b9-28fe-4cc2-8a46-e11ac9401b5a service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:39.850581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc82b9-28fe-4cc2-8a46-e11ac9401b5a service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:39.907819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:39.946847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:39.947536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:39.947880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:39.948224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] PATH_INFO: `/v3/users/8a8136e4709d4e71823ad0e92f31633b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:39.954449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8136e4709d4e71823ad0e92f31633b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:39.954449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:40.017663 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8136e4709d4e71823ad0e92f31633b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.044117 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8136e4709d4e71823ad0e92f31633b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.044822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8136e4709d4e71823ad0e92f31633b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.055971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8136e4709d4e71823ad0e92f31633b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.072210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8136e4709d4e71823ad0e92f31633b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.080711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1226/2477] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:39 2022] DELETE /identity/v3/users/8a8136e4709d4e71823ad0e92f31633b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:40.397375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:40.399078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:40.421176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0c0a-7645-43ef-aad6-d491920a3d6d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:40.498010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:40.543105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:40.554827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:40.555044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:40.555548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:40.556641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] PATH_INFO: `/v3/projects/86d0b9cf1e8a4fd69dda845935a5ad27` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:40.557050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:40.558753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d0b9cf1e8a4fd69dda845935a5ad27)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:40.562536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:40.650899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d0b9cf1e8a4fd69dda845935a5ad27', 'request_id': 'req-b21f247c-668c-4fbf-bfd0-b56e9f59c957'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.658032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d0b9cf1e8a4fd69dda845935a5ad27', 'request_id': 'req-b21f247c-668c-4fbf-bfd0-b56e9f59c957'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.666872 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d0b9cf1e8a4fd69dda845935a5ad27', 'request_id': 'req-b21f247c-668c-4fbf-bfd0-b56e9f59c957'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.679292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bcdb0-819e-471c-a7a6-c1a32a91ee93 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:40.683492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] The token cache is being invalidate because project 86d0b9cf1e8a4fd69dda845935a5ad27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:40.683789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.685122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b21f247c-668c-4fbf-bfd0-b56e9f59c957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:40.689476 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1252/2478] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:40 2022] DELETE /identity/v3/projects/86d0b9cf1e8a4fd69dda845935a5ad27 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:40.693401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a11a57-a1de-4b31-8e81-1b80f0dd668d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:40.693890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a11a57-a1de-4b31-8e81-1b80f0dd668d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:40.694170 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a11a57-a1de-4b31-8e81-1b80f0dd668d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:40.725398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-86a11a57-a1de-4b31-8e81-1b80f0dd668d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:40.726686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-86a11a57-a1de-4b31-8e81-1b80f0dd668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:40.775950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:40.776585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:40.776861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:40.777132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:40.779078 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:40.800751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:40.834574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1253/2479] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:40.844164 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ba04-5b8b-4d92-90df-2dfb0aa7b1f6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:40.844449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ba04-5b8b-4d92-90df-2dfb0aa7b1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:40.844722 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ba04-5b8b-4d92-90df-2dfb0aa7b1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:40.865486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b188ba04-5b8b-4d92-90df-2dfb0aa7b1f6 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:40.866587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b188ba04-5b8b-4d92-90df-2dfb0aa7b1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:40.875702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1254/2480] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:40.879757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188ba04-5b8b-4d92-90df-2dfb0aa7b1f6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:40.880564 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:40.882185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:40.889617 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188ba04-5b8b-4d92-90df-2dfb0aa7b1f6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:40.895302 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:40.896092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:40.896367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:40.896639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:40.897702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:40.897702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:40.899985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:40.910846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1227/2481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:40.920780 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1255/2482] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:44:40.923842 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:40.940865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a01fdd-bdd6-4f0d-a069-8bbe454449f2 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:40.956483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b093a-2612-42a0-bdeb-ae7543372c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:40.957861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b093a-2612-42a0-bdeb-ae7543372c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:40.959157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b093a-2612-42a0-bdeb-ae7543372c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:40.960451 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b093a-2612-42a0-bdeb-ae7543372c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:40.972792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b093a-2612-42a0-bdeb-ae7543372c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:40.972792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b093a-2612-42a0-bdeb-ae7543372c8c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.006694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1228/2483] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:40 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:41.010942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:41.024501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9d037-100a-4035-99f4-ca9b68ca462d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:41.118400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:41.132105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49f56e-d6a4-4bfa-872a-f4c7fc0df959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:41.133394 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49f56e-d6a4-4bfa-872a-f4c7fc0df959 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.133819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49f56e-d6a4-4bfa-872a-f4c7fc0df959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.134460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49f56e-d6a4-4bfa-872a-f4c7fc0df959 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.135801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49f56e-d6a4-4bfa-872a-f4c7fc0df959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:41.142205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49f56e-d6a4-4bfa-872a-f4c7fc0df959 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.188356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1256/2484] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:41 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:41.195833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b093a-2612-42a0-bdeb-ae7543372c8c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:41.216664 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b093a-2612-42a0-bdeb-ae7543372c8c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:41.229252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd764ee-b1c9-4a3a-ab24-4cd021b3d6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:41.230081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd764ee-b1c9-4a3a-ab24-4cd021b3d6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.230538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd764ee-b1c9-4a3a-ab24-4cd021b3d6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.231711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd764ee-b1c9-4a3a-ab24-4cd021b3d6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.232562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd764ee-b1c9-4a3a-ab24-4cd021b3d6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:41.234571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd764ee-b1c9-4a3a-ab24-4cd021b3d6ec admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.257437 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1229/2485] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:41.259325 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49f56e-d6a4-4bfa-872a-f4c7fc0df959 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:41.273959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49f56e-d6a4-4bfa-872a-f4c7fc0df959 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:41.282641 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3ea06-1b0d-48ce-a664-688a95327dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:41.283441 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3ea06-1b0d-48ce-a664-688a95327dde admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.283961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3ea06-1b0d-48ce-a664-688a95327dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.284566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3ea06-1b0d-48ce-a664-688a95327dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.285006 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3ea06-1b0d-48ce-a664-688a95327dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:41.287161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3ea06-1b0d-48ce-a664-688a95327dde admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.309988 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1257/2486] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:41.310644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd764ee-b1c9-4a3a-ab24-4cd021b3d6ec admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:41.322046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd764ee-b1c9-4a3a-ab24-4cd021b3d6ec admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:41.333932 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a029e4-01a6-4e9a-898d-47ca3a4fd25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:41.334540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a029e4-01a6-4e9a-898d-47ca3a4fd25b admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.339708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a029e4-01a6-4e9a-898d-47ca3a4fd25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.339708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a029e4-01a6-4e9a-898d-47ca3a4fd25b admin admin] PATH_INFO: `/v3/projects/fea0980f111b4f5cbd078e3a3439956f/users/c217fb6fed024ca481f4f805003e5c17/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.361505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a029e4-01a6-4e9a-898d-47ca3a4fd25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea0980f111b4f5cbd078e3a3439956f, user_id=c217fb6fed024ca481f4f805003e5c17, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:41.366452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a029e4-01a6-4e9a-898d-47ca3a4fd25b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.387393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1230/2487] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:41 2022] PUT /identity/v3/projects/fea0980f111b4f5cbd078e3a3439956f/users/c217fb6fed024ca481f4f805003e5c17/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:41.398327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9754-326b-4098-b6b3-9dcc347f6237 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.398604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9754-326b-4098-b6b3-9dcc347f6237 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.398878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a9754-326b-4098-b6b3-9dcc347f6237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.431585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-289a9754-326b-4098-b6b3-9dcc347f6237 None None] MFA Rules not processed for user `c217fb6fed024ca481f4f805003e5c17`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:41.434431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-289a9754-326b-4098-b6b3-9dcc347f6237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:41.536169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1258/2488] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:41.539753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a029e4-01a6-4e9a-898d-47ca3a4fd25b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:41.674258 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a029e4-01a6-4e9a-898d-47ca3a4fd25b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:41.680609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8af1bb-ef62-43e2-90ec-6dc8d4402a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:41.681319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8af1bb-ef62-43e2-90ec-6dc8d4402a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.681596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8af1bb-ef62-43e2-90ec-6dc8d4402a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.681864 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8af1bb-ef62-43e2-90ec-6dc8d4402a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.682497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8af1bb-ef62-43e2-90ec-6dc8d4402a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:41.684510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8af1bb-ef62-43e2-90ec-6dc8d4402a8f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.717181 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1231/2489] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:41 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:41.720944 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a9754-326b-4098-b6b3-9dcc347f6237 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:41.733518 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a9754-326b-4098-b6b3-9dcc347f6237 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:41.809975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc6d5c-524a-437d-8d13-cc4f55299e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:41.811400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc6d5c-524a-437d-8d13-cc4f55299e2b admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.811753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc6d5c-524a-437d-8d13-cc4f55299e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.812095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc6d5c-524a-437d-8d13-cc4f55299e2b admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.812852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc6d5c-524a-437d-8d13-cc4f55299e2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:41.815905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc6d5c-524a-437d-8d13-cc4f55299e2b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.847571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1259/2490] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:41 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:41.851450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8af1bb-ef62-43e2-90ec-6dc8d4402a8f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:41.866651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8af1bb-ef62-43e2-90ec-6dc8d4402a8f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:41.874452 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea28f6-2fa5-48ce-858e-c2c16903d036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:41.875943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea28f6-2fa5-48ce-858e-c2c16903d036 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.876305 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea28f6-2fa5-48ce-858e-c2c16903d036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.876641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea28f6-2fa5-48ce-858e-c2c16903d036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.877951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea28f6-2fa5-48ce-858e-c2c16903d036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:41.880636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea28f6-2fa5-48ce-858e-c2c16903d036 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.900635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1232/2491] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:41.905049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc6d5c-524a-437d-8d13-cc4f55299e2b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:41.924927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc6d5c-524a-437d-8d13-cc4f55299e2b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:41.930306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcd910-d901-493e-bd71-4a07b1a0fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:41.931788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcd910-d901-493e-bd71-4a07b1a0fabe admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:41.931788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcd910-d901-493e-bd71-4a07b1a0fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:41.931788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcd910-d901-493e-bd71-4a07b1a0fabe admin admin] PATH_INFO: `/v3/projects/62c4bf028df84017a00f2da46942a58a/users/32c4ac1a3ea34339bafab0e4a61eb1fe/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:41.954126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcd910-d901-493e-bd71-4a07b1a0fabe admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c4bf028df84017a00f2da46942a58a, user_id=32c4ac1a3ea34339bafab0e4a61eb1fe, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:41.957669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcd910-d901-493e-bd71-4a07b1a0fabe admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:41.984556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1260/2492] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:41 2022] PUT /identity/v3/projects/62c4bf028df84017a00f2da46942a58a/users/32c4ac1a3ea34339bafab0e4a61eb1fe/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:41.985585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea28f6-2fa5-48ce-858e-c2c16903d036 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:42.010505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea28f6-2fa5-48ce-858e-c2c16903d036 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:42.065411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:42.117099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797a45f-d5b2-4a71-9f43-cd4ff8593ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:42.117918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a45f-d5b2-4a71-9f43-cd4ff8593ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:42.118200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a45f-d5b2-4a71-9f43-cd4ff8593ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:42.118761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a45f-d5b2-4a71-9f43-cd4ff8593ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:42.119642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797a45f-d5b2-4a71-9f43-cd4ff8593ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:42.122477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797a45f-d5b2-4a71-9f43-cd4ff8593ff6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:42.140991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1233/2493] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:42.142286 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcd910-d901-493e-bd71-4a07b1a0fabe admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:42.158215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcd910-d901-493e-bd71-4a07b1a0fabe admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:42.171911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:42.172600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:42.173273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:42.173273 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] PATH_INFO: `/v3/projects/62c4bf028df84017a00f2da46942a58a/users/32c4ac1a3ea34339bafab0e4a61eb1fe/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:42.178781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c4bf028df84017a00f2da46942a58a, user_id=32c4ac1a3ea34339bafab0e4a61eb1fe, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:42.181580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:42.197289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1261/2494] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:42 2022] PUT /identity/v3/projects/62c4bf028df84017a00f2da46942a58a/users/32c4ac1a3ea34339bafab0e4a61eb1fe/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:42.207453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266852-8f27-47ef-8230-6155b3144a2e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:42.207803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266852-8f27-47ef-8230-6155b3144a2e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:42.208146 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266852-8f27-47ef-8230-6155b3144a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:42.270429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f6266852-8f27-47ef-8230-6155b3144a2e None None] MFA Rules not processed for user `32c4ac1a3ea34339bafab0e4a61eb1fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:42.271807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f6266852-8f27-47ef-8230-6155b3144a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:42.400411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1234/2495] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:42.414642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:42.419094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:42.461571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:42.547245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be1bd2-5f19-490c-8b97-6ad041db42dd admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:42.552285 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:42.553064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:42.553613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:42.553955 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:42.583106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:42.584584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:42.612451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1262/2496] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:42.664952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6266852-8f27-47ef-8230-6155b3144a2e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:42.666521 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f6266852-8f27-47ef-8230-6155b3144a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:42.684404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6266852-8f27-47ef-8230-6155b3144a2e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:42.689197 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d081a-0b6d-4dc8-b8c6-0941b60b90bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:42.689910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d081a-0b6d-4dc8-b8c6-0941b60b90bd service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:42.690226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d081a-0b6d-4dc8-b8c6-0941b60b90bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:42.690618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d081a-0b6d-4dc8-b8c6-0941b60b90bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:42.751214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:42.752631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:42.775574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d081a-0b6d-4dc8-b8c6-0941b60b90bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:42.781267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d081a-0b6d-4dc8-b8c6-0941b60b90bd service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:42.798871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1235/2497] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:42.833317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5a40a-fafb-405a-9c6d-227695673b23 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:42.838271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:42.838916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:42.839191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:42.839457 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] PATH_INFO: `/v3/users/289c7d8b89cd445a924f5ac07e85fe48` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:42.841392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=289c7d8b89cd445a924f5ac07e85fe48)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:42.843313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:42.905367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289c7d8b89cd445a924f5ac07e85fe48'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:42.914384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289c7d8b89cd445a924f5ac07e85fe48'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:42.915296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289c7d8b89cd445a924f5ac07e85fe48'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:42.931106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289c7d8b89cd445a924f5ac07e85fe48'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:42.932132 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289c7d8b89cd445a924f5ac07e85fe48'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:42.948534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1263/2498] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:42 2022] DELETE /identity/v3/users/289c7d8b89cd445a924f5ac07e85fe48 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:43.159592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d081a-0b6d-4dc8-b8c6-0941b60b90bd service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:43.177615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d081a-0b6d-4dc8-b8c6-0941b60b90bd service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:43.263844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:43.284518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:43.285505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.285840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.286196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] PATH_INFO: `/v3/projects/6f889848d459459484b5b5517fef576b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.290446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f889848d459459484b5b5517fef576b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:43.292383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:43.297624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:43.299563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:43.344085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f889848d459459484b5b5517fef576b', 'request_id': 'req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:43.351400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:43.352606 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f889848d459459484b5b5517fef576b', 'request_id': 'req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:43.361251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f889848d459459484b5b5517fef576b', 'request_id': 'req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:43.376516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] The token cache is being invalidate because project 6f889848d459459484b5b5517fef576b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:43.376844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:43.377570 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91fe1ad0-f1c4-4769-89e8-72bdeb0e256a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:43.382530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1236/2499] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:43 2022] DELETE /identity/v3/projects/6f889848d459459484b5b5517fef576b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:43.384252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cd23d-ebe7-4784-90e0-361ffcb3a07d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.384534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cd23d-ebe7-4784-90e0-361ffcb3a07d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.384805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cd23d-ebe7-4784-90e0-361ffcb3a07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.396225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352912c-9333-4cea-b1c9-fa2dca417bcc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:43.410887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f80cd23d-ebe7-4784-90e0-361ffcb3a07d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:43.412165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f80cd23d-ebe7-4784-90e0-361ffcb3a07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:43.478298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:43.479015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.479359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.479702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.482177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:43.482599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1237/2500] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:43.490328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceb180-f395-4430-80f5-10ddbb39bef0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.490645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceb180-f395-4430-80f5-10ddbb39bef0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.490938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceb180-f395-4430-80f5-10ddbb39bef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.495440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:43.497121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:43.509117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1264/2501] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:44:43.514287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-28ceb180-f395-4430-80f5-10ddbb39bef0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:43.515645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-28ceb180-f395-4430-80f5-10ddbb39bef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:43.525660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1238/2502] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:43.531506 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:43.544582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fbc0a-c658-4fe8-b5e5-936c933f44f2 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:43.549915 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:43.550652 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.551001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.551246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.552064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:43.552369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:43.554121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:43.567581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1265/2503] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:44:43.571801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceb180-f395-4430-80f5-10ddbb39bef0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:43.595024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceb180-f395-4430-80f5-10ddbb39bef0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:43.601118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ce070-9570-4633-b36e-b98f65bcebdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:43.601795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ce070-9570-4633-b36e-b98f65bcebdc admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.602072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ce070-9570-4633-b36e-b98f65bcebdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.602346 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ce070-9570-4633-b36e-b98f65bcebdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.602973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ce070-9570-4633-b36e-b98f65bcebdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:43.604941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ce070-9570-4633-b36e-b98f65bcebdc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:43.646677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1239/2504] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:43 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:43.647762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:43.659604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6fab0-035a-412c-82f6-6ada6dee5c89 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:43.736179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f93449-cd2f-4f9e-9abd-caa9fd3f1d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:43.736800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f93449-cd2f-4f9e-9abd-caa9fd3f1d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.737100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f93449-cd2f-4f9e-9abd-caa9fd3f1d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.737485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f93449-cd2f-4f9e-9abd-caa9fd3f1d4d admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.738359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f93449-cd2f-4f9e-9abd-caa9fd3f1d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:43.740343 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f93449-cd2f-4f9e-9abd-caa9fd3f1d4d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:43.770035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1266/2505] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:43 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:43.771368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ce070-9570-4633-b36e-b98f65bcebdc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:43.785431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ce070-9570-4633-b36e-b98f65bcebdc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:43.790590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04178-2e84-4a0d-ae32-542fd8f65b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:43.791238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04178-2e84-4a0d-ae32-542fd8f65b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.791508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04178-2e84-4a0d-ae32-542fd8f65b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.792746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04178-2e84-4a0d-ae32-542fd8f65b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.792746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee04178-2e84-4a0d-ae32-542fd8f65b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:43.795083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee04178-2e84-4a0d-ae32-542fd8f65b31 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:43.808102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1240/2506] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:43.813708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f93449-cd2f-4f9e-9abd-caa9fd3f1d4d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:43.827317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f93449-cd2f-4f9e-9abd-caa9fd3f1d4d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:43.832688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405da58-020b-486c-b1ee-f43a57ace327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:43.833338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405da58-020b-486c-b1ee-f43a57ace327 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.833614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405da58-020b-486c-b1ee-f43a57ace327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.833881 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405da58-020b-486c-b1ee-f43a57ace327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.834539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405da58-020b-486c-b1ee-f43a57ace327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:43.836161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405da58-020b-486c-b1ee-f43a57ace327 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:43.849210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1267/2507] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:43.853515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04178-2e84-4a0d-ae32-542fd8f65b31 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:43.864806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04178-2e84-4a0d-ae32-542fd8f65b31 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:43.869659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be1125-0cb6-40c7-80b7-aa9e8eb90092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:43.870316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be1125-0cb6-40c7-80b7-aa9e8eb90092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.870591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be1125-0cb6-40c7-80b7-aa9e8eb90092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.870857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be1125-0cb6-40c7-80b7-aa9e8eb90092 admin admin] PATH_INFO: `/v3/projects/145b73a4868e4b50a6ed18ef69128f3c/users/903ecebf5c9743118fb5eb2c71b949a5/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.888147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be1125-0cb6-40c7-80b7-aa9e8eb90092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145b73a4868e4b50a6ed18ef69128f3c, user_id=903ecebf5c9743118fb5eb2c71b949a5, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:43.891443 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be1125-0cb6-40c7-80b7-aa9e8eb90092 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:43.915358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1241/2508] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:43 2022] PUT /identity/v3/projects/145b73a4868e4b50a6ed18ef69128f3c/users/903ecebf5c9743118fb5eb2c71b949a5/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:43.927370 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66b1e9-ab73-4636-9bcd-2283132c34ee None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:43.927656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66b1e9-ab73-4636-9bcd-2283132c34ee None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:43.927929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66b1e9-ab73-4636-9bcd-2283132c34ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:43.958764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9f66b1e9-ab73-4636-9bcd-2283132c34ee None None] MFA Rules not processed for user `903ecebf5c9743118fb5eb2c71b949a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:43.960084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9f66b1e9-ab73-4636-9bcd-2283132c34ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:43.983580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:44.044363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1268/2509] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:44.048683 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be1125-0cb6-40c7-80b7-aa9e8eb90092 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:44.069789 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:44.129889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be1125-0cb6-40c7-80b7-aa9e8eb90092 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:44.138562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae602-c466-4d07-b4b9-b1bc7b24832e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:44.139195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae602-c466-4d07-b4b9-b1bc7b24832e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.139975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae602-c466-4d07-b4b9-b1bc7b24832e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.140183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae602-c466-4d07-b4b9-b1bc7b24832e admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.140829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ae602-c466-4d07-b4b9-b1bc7b24832e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:44.143519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ae602-c466-4d07-b4b9-b1bc7b24832e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:44.191209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1242/2510] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:44 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:44.194784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66b1e9-ab73-4636-9bcd-2283132c34ee None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:44.206234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66b1e9-ab73-4636-9bcd-2283132c34ee None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:44.283481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2595152-6d95-476a-8701-d79f0594301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:44.284153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595152-6d95-476a-8701-d79f0594301d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.284397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595152-6d95-476a-8701-d79f0594301d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.284667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595152-6d95-476a-8701-d79f0594301d admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.285327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2595152-6d95-476a-8701-d79f0594301d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:44.287268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2595152-6d95-476a-8701-d79f0594301d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:44.314634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1269/2511] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:44 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:44.318082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae602-c466-4d07-b4b9-b1bc7b24832e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:44.334218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae602-c466-4d07-b4b9-b1bc7b24832e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:44.340566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114f201-8960-422c-bbf3-f33104f89f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:44.341391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114f201-8960-422c-bbf3-f33104f89f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.341754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114f201-8960-422c-bbf3-f33104f89f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.342033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114f201-8960-422c-bbf3-f33104f89f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.342697 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114f201-8960-422c-bbf3-f33104f89f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:44.344307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114f201-8960-422c-bbf3-f33104f89f6c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:44.361275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1243/2512] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:44.364485 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2595152-6d95-476a-8701-d79f0594301d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:44.375612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2595152-6d95-476a-8701-d79f0594301d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:44.380788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a572520-8b9a-4785-bb93-92c446985be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:44.381703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a572520-8b9a-4785-bb93-92c446985be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.381986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a572520-8b9a-4785-bb93-92c446985be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.382260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a572520-8b9a-4785-bb93-92c446985be9 admin admin] PATH_INFO: `/v3/projects/14bc2dad41104fb284535aef4befb504/users/12596bac740142a99724a4bfd119bc77/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.402020 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a572520-8b9a-4785-bb93-92c446985be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14bc2dad41104fb284535aef4befb504, user_id=12596bac740142a99724a4bfd119bc77, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:44.404645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a572520-8b9a-4785-bb93-92c446985be9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:44.437736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1270/2513] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:44 2022] PUT /identity/v3/projects/14bc2dad41104fb284535aef4befb504/users/12596bac740142a99724a4bfd119bc77/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:44.441884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114f201-8960-422c-bbf3-f33104f89f6c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:44.452854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114f201-8960-422c-bbf3-f33104f89f6c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:44.531811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9773c9-8614-4dbf-b380-648a9a71580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:44.532433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9773c9-8614-4dbf-b380-648a9a71580e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.532705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9773c9-8614-4dbf-b380-648a9a71580e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.532969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9773c9-8614-4dbf-b380-648a9a71580e admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.533650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9773c9-8614-4dbf-b380-648a9a71580e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:44.535254 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9773c9-8614-4dbf-b380-648a9a71580e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:44.547788 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1244/2514] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:44.552821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a572520-8b9a-4785-bb93-92c446985be9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:44.564914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a572520-8b9a-4785-bb93-92c446985be9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:44.570310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:44.570974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.571245 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.571515 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] PATH_INFO: `/v3/projects/14bc2dad41104fb284535aef4befb504/users/12596bac740142a99724a4bfd119bc77/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.573978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14bc2dad41104fb284535aef4befb504, user_id=12596bac740142a99724a4bfd119bc77, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:44.576817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:44.588812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1271/2515] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:44 2022] PUT /identity/v3/projects/14bc2dad41104fb284535aef4befb504/users/12596bac740142a99724a4bfd119bc77/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:44.602304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7861a8-d7fe-49d0-8fd6-d4df4da2ae82 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.602884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7861a8-d7fe-49d0-8fd6-d4df4da2ae82 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.603314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7861a8-d7fe-49d0-8fd6-d4df4da2ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.638887 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ec7861a8-d7fe-49d0-8fd6-d4df4da2ae82 None None] MFA Rules not processed for user `12596bac740142a99724a4bfd119bc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:44.640557 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ec7861a8-d7fe-49d0-8fd6-d4df4da2ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:44.728440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1245/2516] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:44.741300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:44.743090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:44.806547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335afd98-5a34-4cb6-a372-8102fc091584 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:44.810941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:44.811653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.812098 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.812367 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.822514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:44.824121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:44.834314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1272/2517] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:44.921843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7861a8-d7fe-49d0-8fd6-d4df4da2ae82 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:44.931448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7861a8-d7fe-49d0-8fd6-d4df4da2ae82 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:44.935111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a2faa-88ce-418a-a50a-196b7f6de72d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:44.935735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a2faa-88ce-418a-a50a-196b7f6de72d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:44.936009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a2faa-88ce-418a-a50a-196b7f6de72d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:44.936276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a2faa-88ce-418a-a50a-196b7f6de72d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:44.945854 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:44.993939 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a2faa-88ce-418a-a50a-196b7f6de72d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:44.995281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a2faa-88ce-418a-a50a-196b7f6de72d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:45.004531 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1246/2518] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:46.475706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:46.478444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:46.506642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:46.638564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e10a1-4a99-499c-9eb0-750a47e37ff2 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:46.643574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:46.644216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:46.644577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:46.644855 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] PATH_INFO: `/v3/users/c217fb6fed024ca481f4f805003e5c17` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:46.646306 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c217fb6fed024ca481f4f805003e5c17)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:46.648366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:46.704752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c217fb6fed024ca481f4f805003e5c17'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:46.716645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c217fb6fed024ca481f4f805003e5c17'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:46.717683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c217fb6fed024ca481f4f805003e5c17'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:46.732222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c217fb6fed024ca481f4f805003e5c17'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:46.732930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c217fb6fed024ca481f4f805003e5c17'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:46.748852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1273/2519] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:46 2022] DELETE /identity/v3/users/c217fb6fed024ca481f4f805003e5c17 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:46.753689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa14d6-9712-4815-8f4c-57826400d251 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:46.754249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa14d6-9712-4815-8f4c-57826400d251 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:46.754249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa14d6-9712-4815-8f4c-57826400d251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:46.775184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a4aa14d6-9712-4815-8f4c-57826400d251 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:46.776399 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a4aa14d6-9712-4815-8f4c-57826400d251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:46.868239 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1247/2520] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:46.878213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:46.880007 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:46.953784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0465d3d-ffed-4abd-9ce9-40b0bae3afa5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:46.958623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff3f67-5975-4c62-96f7-f6bc3e2707fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:46.959357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff3f67-5975-4c62-96f7-f6bc3e2707fe service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:46.959731 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff3f67-5975-4c62-96f7-f6bc3e2707fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:46.960113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff3f67-5975-4c62-96f7-f6bc3e2707fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:46.972345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff3f67-5975-4c62-96f7-f6bc3e2707fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:46.974246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff3f67-5975-4c62-96f7-f6bc3e2707fe service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:46.988820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1274/2521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:47.284057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa14d6-9712-4815-8f4c-57826400d251 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:47.285571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a4aa14d6-9712-4815-8f4c-57826400d251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:47.301922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa14d6-9712-4815-8f4c-57826400d251 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:47.307633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:47.309493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:47.309780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:47.310071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] PATH_INFO: `/v3/projects/fea0980f111b4f5cbd078e3a3439956f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:47.311243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea0980f111b4f5cbd078e3a3439956f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:47.313212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:47.364580 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea0980f111b4f5cbd078e3a3439956f', 'request_id': 'req-e893b02e-7a10-4dad-b125-44667a3bb556'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.370529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea0980f111b4f5cbd078e3a3439956f', 'request_id': 'req-e893b02e-7a10-4dad-b125-44667a3bb556'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.378423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea0980f111b4f5cbd078e3a3439956f', 'request_id': 'req-e893b02e-7a10-4dad-b125-44667a3bb556'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.391134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] The token cache is being invalidate because project fea0980f111b4f5cbd078e3a3439956f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:47.391477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.392103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.393605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1248/2522] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:47 2022] DELETE /identity/v3/projects/fea0980f111b4f5cbd078e3a3439956f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:47.397717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff3f67-5975-4c62-96f7-f6bc3e2707fe service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:47.408444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff3f67-5975-4c62-96f7-f6bc3e2707fe service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:47.441638 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:47.470945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:47.471654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:47.471861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:47.472137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] PATH_INFO: `/v3/users/32c4ac1a3ea34339bafab0e4a61eb1fe` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:47.473546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c4ac1a3ea34339bafab0e4a61eb1fe)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:47.475481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:47.527092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c4ac1a3ea34339bafab0e4a61eb1fe'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.540134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c4ac1a3ea34339bafab0e4a61eb1fe'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.540134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c4ac1a3ea34339bafab0e4a61eb1fe'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.551982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c4ac1a3ea34339bafab0e4a61eb1fe'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.552582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-70040e5c-436f-43cf-852f-1addb0ba8bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c4ac1a3ea34339bafab0e4a61eb1fe'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:47.565751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1275/2523] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:47 2022] DELETE /identity/v3/users/32c4ac1a3ea34339bafab0e4a61eb1fe => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:47.875242 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:47.878901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:47.902006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893b02e-7a10-4dad-b125-44667a3bb556 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:47.948127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:47.974762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:47.975391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:47.975667 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:47.975935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] PATH_INFO: `/v3/projects/62c4bf028df84017a00f2da46942a58a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:47.977022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c4bf028df84017a00f2da46942a58a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:47.978997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.024733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c4bf028df84017a00f2da46942a58a', 'request_id': 'req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:48.030850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c4bf028df84017a00f2da46942a58a', 'request_id': 'req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:48.037086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c4bf028df84017a00f2da46942a58a', 'request_id': 'req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:48.050958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] The token cache is being invalidate because project 62c4bf028df84017a00f2da46942a58a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:48.051247 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:48.051882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1e4ba126-f815-4b07-8cb8-e0ec6a98c5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:48.058722 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1249/2524] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:47 2022] DELETE /identity/v3/projects/62c4bf028df84017a00f2da46942a58a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:48.063411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3281-dc4f-4a64-b611-cfe92a5ef830 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.063697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3281-dc4f-4a64-b611-cfe92a5ef830 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.064029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3281-dc4f-4a64-b611-cfe92a5ef830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.083802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8eeb3281-dc4f-4a64-b611-cfe92a5ef830 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:48.085090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8eeb3281-dc4f-4a64-b611-cfe92a5ef830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:48.161399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1276/2525] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:48.163174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7810f-8808-47d2-a85d-718c037478b9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.163456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7810f-8808-47d2-a85d-718c037478b9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.163730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7810f-8808-47d2-a85d-718c037478b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.186417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-98d7810f-8808-47d2-a85d-718c037478b9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:48.187665 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-98d7810f-8808-47d2-a85d-718c037478b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:48.197208 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1250/2526] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:48.203822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb3281-dc4f-4a64-b611-cfe92a5ef830 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.214356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb3281-dc4f-4a64-b611-cfe92a5ef830 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.219848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.220454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.220763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.221030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.221714 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:48.222011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.223687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.238982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1277/2527] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:44:48.243635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7810f-8808-47d2-a85d-718c037478b9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.254649 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7810f-8808-47d2-a85d-718c037478b9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.259152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab727-90ac-4291-b012-103e2d39a33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.259747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab727-90ac-4291-b012-103e2d39a33a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.260020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab727-90ac-4291-b012-103e2d39a33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.260282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab727-90ac-4291-b012-103e2d39a33a admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.260920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eab727-90ac-4291-b012-103e2d39a33a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.263371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eab727-90ac-4291-b012-103e2d39a33a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.290510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1251/2528] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:48 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:48.294740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.305301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054c980-10eb-465f-ae64-707dbfed9f29 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.344690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:48.363510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe8d0-97c6-4274-952d-eae2fd2b9947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.364180 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe8d0-97c6-4274-952d-eae2fd2b9947 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.364516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe8d0-97c6-4274-952d-eae2fd2b9947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.364784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe8d0-97c6-4274-952d-eae2fd2b9947 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.365435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fe8d0-97c6-4274-952d-eae2fd2b9947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.367483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fe8d0-97c6-4274-952d-eae2fd2b9947 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.389493 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1278/2529] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:48 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:48.393220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab727-90ac-4291-b012-103e2d39a33a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.404463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab727-90ac-4291-b012-103e2d39a33a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.409337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fcae7-f282-4c09-a72b-a6b0c5d4283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.409940 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fcae7-f282-4c09-a72b-a6b0c5d4283a admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.410214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fcae7-f282-4c09-a72b-a6b0c5d4283a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.410480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fcae7-f282-4c09-a72b-a6b0c5d4283a admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.411116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fcae7-f282-4c09-a72b-a6b0c5d4283a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.412754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fcae7-f282-4c09-a72b-a6b0c5d4283a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.422821 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1252/2530] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:48.426910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe8d0-97c6-4274-952d-eae2fd2b9947 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.437069 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe8d0-97c6-4274-952d-eae2fd2b9947 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.441741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6026ea-619b-4278-bfb9-41125cca774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.442329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6026ea-619b-4278-bfb9-41125cca774b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.442605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6026ea-619b-4278-bfb9-41125cca774b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.442875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6026ea-619b-4278-bfb9-41125cca774b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.443510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6026ea-619b-4278-bfb9-41125cca774b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.445120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6026ea-619b-4278-bfb9-41125cca774b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.456580 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1279/2531] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:48.460655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fcae7-f282-4c09-a72b-a6b0c5d4283a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.472471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fcae7-f282-4c09-a72b-a6b0c5d4283a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.477036 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb537-dddb-440a-b9c3-59d6022f2300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.477750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb537-dddb-440a-b9c3-59d6022f2300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.478020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb537-dddb-440a-b9c3-59d6022f2300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.478284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb537-dddb-440a-b9c3-59d6022f2300 admin admin] PATH_INFO: `/v3/projects/ad8333030f5d41c2aba43ed831552f9a/users/7bbffc5b1d2242c4929c63ca3560d46c/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.493560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bb537-dddb-440a-b9c3-59d6022f2300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8333030f5d41c2aba43ed831552f9a, user_id=7bbffc5b1d2242c4929c63ca3560d46c, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.496359 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bb537-dddb-440a-b9c3-59d6022f2300 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.513559 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1253/2532] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:48 2022] PUT /identity/v3/projects/ad8333030f5d41c2aba43ed831552f9a/users/7bbffc5b1d2242c4929c63ca3560d46c/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:48.523453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e0069-e207-40af-814f-29e2c479e7c7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.523726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e0069-e207-40af-814f-29e2c479e7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.524013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e0069-e207-40af-814f-29e2c479e7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.551584 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-834e0069-e207-40af-814f-29e2c479e7c7 None None] MFA Rules not processed for user `7bbffc5b1d2242c4929c63ca3560d46c`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:48.552777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-834e0069-e207-40af-814f-29e2c479e7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:48.617655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1280/2533] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:48.623871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb537-dddb-440a-b9c3-59d6022f2300 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.685357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb537-dddb-440a-b9c3-59d6022f2300 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.689934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296aca9-4ea8-43e1-9073-f5c6c9dbd577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.690542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296aca9-4ea8-43e1-9073-f5c6c9dbd577 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.690825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296aca9-4ea8-43e1-9073-f5c6c9dbd577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.691084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296aca9-4ea8-43e1-9073-f5c6c9dbd577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.691694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296aca9-4ea8-43e1-9073-f5c6c9dbd577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.693612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296aca9-4ea8-43e1-9073-f5c6c9dbd577 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.699402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:48.723853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1254/2534] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:48 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:48.728145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e0069-e207-40af-814f-29e2c479e7c7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.739726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e0069-e207-40af-814f-29e2c479e7c7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.798868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c55c25-2dcf-4fdf-8186-9baa926c0679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.799480 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c55c25-2dcf-4fdf-8186-9baa926c0679 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.799912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c55c25-2dcf-4fdf-8186-9baa926c0679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.800189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c55c25-2dcf-4fdf-8186-9baa926c0679 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.800814 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c55c25-2dcf-4fdf-8186-9baa926c0679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.802807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c55c25-2dcf-4fdf-8186-9baa926c0679 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.825484 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1281/2535] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:48 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:48.829306 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296aca9-4ea8-43e1-9073-f5c6c9dbd577 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.839403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296aca9-4ea8-43e1-9073-f5c6c9dbd577 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.844158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8705a-2506-46ec-b2eb-56cf76977731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.844761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8705a-2506-46ec-b2eb-56cf76977731 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.845035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8705a-2506-46ec-b2eb-56cf76977731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.845340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8705a-2506-46ec-b2eb-56cf76977731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.846006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8705a-2506-46ec-b2eb-56cf76977731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.848843 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8705a-2506-46ec-b2eb-56cf76977731 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.862130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1255/2536] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:48.863812 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c55c25-2dcf-4fdf-8186-9baa926c0679 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.874329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c55c25-2dcf-4fdf-8186-9baa926c0679 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.879368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3a54-3cb4-4a07-a0bd-0c216530b636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.879831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3a54-3cb4-4a07-a0bd-0c216530b636 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.880105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3a54-3cb4-4a07-a0bd-0c216530b636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.880374 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3a54-3cb4-4a07-a0bd-0c216530b636 admin admin] PATH_INFO: `/v3/projects/f4a75bf1f75842cb81c245a6173bdf21/users/8fe2c7eb4459424896a6d5722d8d2b02/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.898323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c3a54-3cb4-4a07-a0bd-0c216530b636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a75bf1f75842cb81c245a6173bdf21, user_id=8fe2c7eb4459424896a6d5722d8d2b02, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.901959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c3a54-3cb4-4a07-a0bd-0c216530b636 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:48.918962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1282/2537] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:48 2022] PUT /identity/v3/projects/f4a75bf1f75842cb81c245a6173bdf21/users/8fe2c7eb4459424896a6d5722d8d2b02/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:48.923179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8705a-2506-46ec-b2eb-56cf76977731 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:48.934325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8705a-2506-46ec-b2eb-56cf76977731 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:48.991314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5f157-5d7c-4954-9010-0e9b0ba5de7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:48.991918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5f157-5d7c-4954-9010-0e9b0ba5de7f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:48.992190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5f157-5d7c-4954-9010-0e9b0ba5de7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:48.992454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5f157-5d7c-4954-9010-0e9b0ba5de7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:48.993075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5f157-5d7c-4954-9010-0e9b0ba5de7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:48.994700 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5f157-5d7c-4954-9010-0e9b0ba5de7f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:49.005185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1256/2538] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:49.009381 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3a54-3cb4-4a07-a0bd-0c216530b636 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:49.019857 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3a54-3cb4-4a07-a0bd-0c216530b636 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:49.024414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:49.025045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:49.025359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:49.025721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] PATH_INFO: `/v3/projects/f4a75bf1f75842cb81c245a6173bdf21/users/8fe2c7eb4459424896a6d5722d8d2b02/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:49.027770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a75bf1f75842cb81c245a6173bdf21, user_id=8fe2c7eb4459424896a6d5722d8d2b02, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:49.031322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:49.041647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1283/2539] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:49 2022] PUT /identity/v3/projects/f4a75bf1f75842cb81c245a6173bdf21/users/8fe2c7eb4459424896a6d5722d8d2b02/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:49.053603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7673ca-8a17-4338-a1a3-fa9ee0239c21 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:49.053934 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7673ca-8a17-4338-a1a3-fa9ee0239c21 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:49.054312 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7673ca-8a17-4338-a1a3-fa9ee0239c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:49.082187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7e7673ca-8a17-4338-a1a3-fa9ee0239c21 None None] MFA Rules not processed for user `8fe2c7eb4459424896a6d5722d8d2b02`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:49.083391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7e7673ca-8a17-4338-a1a3-fa9ee0239c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:49.151013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1257/2540] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:49.161949 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:49.163691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:49.169401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:49.222312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b58653-fd89-4790-98e2-11d11c72515a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:49.225817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377328e-f1c3-4108-ac3e-e6f6f11917c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:49.226413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377328e-f1c3-4108-ac3e-e6f6f11917c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:49.226687 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377328e-f1c3-4108-ac3e-e6f6f11917c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:49.226954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377328e-f1c3-4108-ac3e-e6f6f11917c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:49.236179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377328e-f1c3-4108-ac3e-e6f6f11917c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:49.237634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377328e-f1c3-4108-ac3e-e6f6f11917c0 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:49.248454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1284/2541] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:49 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:49.304402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7673ca-8a17-4338-a1a3-fa9ee0239c21 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:49.305813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7e7673ca-8a17-4338-a1a3-fa9ee0239c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:49.317779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7673ca-8a17-4338-a1a3-fa9ee0239c21 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:49.321373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae2aa5-6a08-4f91-827b-1c15320bd8da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:49.321968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae2aa5-6a08-4f91-827b-1c15320bd8da service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:49.322244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae2aa5-6a08-4f91-827b-1c15320bd8da service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:49.322509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae2aa5-6a08-4f91-827b-1c15320bd8da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:49.375566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:49.378248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae2aa5-6a08-4f91-827b-1c15320bd8da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:49.380159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae2aa5-6a08-4f91-827b-1c15320bd8da service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:49.390309 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1258/2542] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:54.189143 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377328e-f1c3-4108-ac3e-e6f6f11917c0 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:54.273977 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377328e-f1c3-4108-ac3e-e6f6f11917c0 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:54.281190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:54.281860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:54.282200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:54.282473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] PATH_INFO: `/v3/users/7bbffc5b1d2242c4929c63ca3560d46c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:54.283654 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bbffc5b1d2242c4929c63ca3560d46c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:54.285777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:54.340841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bbffc5b1d2242c4929c63ca3560d46c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.348546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bbffc5b1d2242c4929c63ca3560d46c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.349073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bbffc5b1d2242c4929c63ca3560d46c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.367458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bbffc5b1d2242c4929c63ca3560d46c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.368220 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bbffc5b1d2242c4929c63ca3560d46c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.381002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1285/2543] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:54 2022] DELETE /identity/v3/users/7bbffc5b1d2242c4929c63ca3560d46c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:54.385690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc786-481b-47f4-96b4-07d4433f3648 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:54.386499 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc786-481b-47f4-96b4-07d4433f3648 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:54.386850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc786-481b-47f4-96b4-07d4433f3648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:54.407954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9bddc786-481b-47f4-96b4-07d4433f3648 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:54.409172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9bddc786-481b-47f4-96b4-07d4433f3648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:54.472162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1259/2544] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:54.482453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:54.484200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:54.516448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:54.553350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87352731-cf66-4f82-be9e-e00843fcfbe3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:54.557251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7d39f-ef8c-4c32-8dc0-fb42191db2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:54.557861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d39f-ef8c-4c32-8dc0-fb42191db2fe service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:54.558137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d39f-ef8c-4c32-8dc0-fb42191db2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:54.558433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d39f-ef8c-4c32-8dc0-fb42191db2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:54.567458 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7d39f-ef8c-4c32-8dc0-fb42191db2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:54.568925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7d39f-ef8c-4c32-8dc0-fb42191db2fe service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:54.578551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1286/2545] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:54.848729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddc786-481b-47f4-96b4-07d4433f3648 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:54.850191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9bddc786-481b-47f4-96b4-07d4433f3648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:54.863269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddc786-481b-47f4-96b4-07d4433f3648 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:54.868024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:54.868792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:54.869112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:54.869370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] PATH_INFO: `/v3/projects/ad8333030f5d41c2aba43ed831552f9a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:54.871571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8333030f5d41c2aba43ed831552f9a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:54.873791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:54.925611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8333030f5d41c2aba43ed831552f9a', 'request_id': 'req-b1fe1018-60bc-468d-b1bd-fa937bc265b8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.930977 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8333030f5d41c2aba43ed831552f9a', 'request_id': 'req-b1fe1018-60bc-468d-b1bd-fa937bc265b8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.937494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8333030f5d41c2aba43ed831552f9a', 'request_id': 'req-b1fe1018-60bc-468d-b1bd-fa937bc265b8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.950537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] The token cache is being invalidate because project ad8333030f5d41c2aba43ed831552f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:54.950813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.951420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:54.952878 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1260/2546] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:54 2022] DELETE /identity/v3/projects/ad8333030f5d41c2aba43ed831552f9a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:54.956850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7d39f-ef8c-4c32-8dc0-fb42191db2fe service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:54.967010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7d39f-ef8c-4c32-8dc0-fb42191db2fe service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:55.027023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:55.028122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.028396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.028662 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] PATH_INFO: `/v3/users/8fe2c7eb4459424896a6d5722d8d2b02` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.029791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fe2c7eb4459424896a6d5722d8d2b02)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:55.031711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:55.076162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fe2c7eb4459424896a6d5722d8d2b02'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.084536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fe2c7eb4459424896a6d5722d8d2b02'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.085145 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fe2c7eb4459424896a6d5722d8d2b02'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.098065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fe2c7eb4459424896a6d5722d8d2b02'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.098763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-187b282b-43cf-4720-aa32-59ad4286e4f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fe2c7eb4459424896a6d5722d8d2b02'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.100191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:55.101628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:55.113126 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1287/2547] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:54 2022] DELETE /identity/v3/users/8fe2c7eb4459424896a6d5722d8d2b02 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:55.119702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe1018-60bc-468d-b1bd-fa937bc265b8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:55.182584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:55.184523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.185167 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.185660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] PATH_INFO: `/v3/users/0a854a8c58714ae7b60639cd25b30220` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.187293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a854a8c58714ae7b60639cd25b30220)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:55.190343 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:55.231064 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:55.265546 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a854a8c58714ae7b60639cd25b30220'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.274137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a854a8c58714ae7b60639cd25b30220'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.275332 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a854a8c58714ae7b60639cd25b30220'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.291267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a854a8c58714ae7b60639cd25b30220'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.306347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a854a8c58714ae7b60639cd25b30220'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.312495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1261/2548] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:55 2022] DELETE /identity/v3/users/0a854a8c58714ae7b60639cd25b30220 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:55.317063 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d55d7e-8c7f-4c73-bbac-3afc8e92a657 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.317533 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d55d7e-8c7f-4c73-bbac-3afc8e92a657 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.317656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d55d7e-8c7f-4c73-bbac-3afc8e92a657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.363179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-93d55d7e-8c7f-4c73-bbac-3afc8e92a657 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:55.365924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-93d55d7e-8c7f-4c73-bbac-3afc8e92a657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:55.419826 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:55.451954 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1288/2549] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:55.464250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:55.466553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:55.477929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d55d7e-8c7f-4c73-bbac-3afc8e92a657 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:55.479762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-93d55d7e-8c7f-4c73-bbac-3afc8e92a657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:55.503161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d55d7e-8c7f-4c73-bbac-3afc8e92a657 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:55.508836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:55.510296 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.510718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.511093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] PATH_INFO: `/v3/projects/f4a75bf1f75842cb81c245a6173bdf21` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.512510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a75bf1f75842cb81c245a6173bdf21)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:55.515596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:55.552381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd69746-3dc1-4ea2-ab04-cd638b205e54 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:55.563031 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aabcc3-6e40-4934-97e8-744055c03880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:55.563710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aabcc3-6e40-4934-97e8-744055c03880 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.563999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aabcc3-6e40-4934-97e8-744055c03880 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.564272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aabcc3-6e40-4934-97e8-744055c03880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.580233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aabcc3-6e40-4934-97e8-744055c03880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:55.583142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aabcc3-6e40-4934-97e8-744055c03880 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:55.592044 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a75bf1f75842cb81c245a6173bdf21', 'request_id': 'req-63b942e5-d565-44a3-af63-f17f846f1576'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.597463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1262/2550] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:55.600694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a75bf1f75842cb81c245a6173bdf21', 'request_id': 'req-63b942e5-d565-44a3-af63-f17f846f1576'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.609479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a75bf1f75842cb81c245a6173bdf21', 'request_id': 'req-63b942e5-d565-44a3-af63-f17f846f1576'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.639016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] The token cache is being invalidate because project f4a75bf1f75842cb81c245a6173bdf21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:55.639750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.640997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-63b942e5-d565-44a3-af63-f17f846f1576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:55.642835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1289/2551] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:55 2022] DELETE /identity/v3/projects/f4a75bf1f75842cb81c245a6173bdf21 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:55.648180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc7ade-abcd-44f2-83a6-a0228c2eb131 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.648471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc7ade-abcd-44f2-83a6-a0228c2eb131 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.648746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc7ade-abcd-44f2-83a6-a0228c2eb131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.671525 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-36bc7ade-abcd-44f2-83a6-a0228c2eb131 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:55.672881 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-36bc7ade-abcd-44f2-83a6-a0228c2eb131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:55.758000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1263/2552] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:55.762141 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b620c1-5391-44a7-b108-99a97c28c19c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.762408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b620c1-5391-44a7-b108-99a97c28c19c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.762817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b620c1-5391-44a7-b108-99a97c28c19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.803078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-05b620c1-5391-44a7-b108-99a97c28c19c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:55.804639 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-05b620c1-5391-44a7-b108-99a97c28c19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:55.814735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1290/2553] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:55.819350 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc7ade-abcd-44f2-83a6-a0228c2eb131 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:55.832144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc7ade-abcd-44f2-83a6-a0228c2eb131 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:55.838516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:55.839149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.839478 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.839739 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.840955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:55.841326 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:55.843153 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:55.856749 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1264/2554] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:44:55.861406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b620c1-5391-44a7-b108-99a97c28c19c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:55.874567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b620c1-5391-44a7-b108-99a97c28c19c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:55.880404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:55.881138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:55.881554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:55.882014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:55.882709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:55.884919 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:55.921198 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1291/2555] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:55 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:55.926461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:55.928412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:55.930972 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:55.941743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54d591-3a1a-4386-971e-e30be0c0fe12 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:55.973828 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:56.003701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94053ac0-1919-4b35-b70f-caad58d10281 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.006466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19590-0088-4d3b-af2f-e8fc855e9ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.007075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19590-0088-4d3b-af2f-e8fc855e9ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.007349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19590-0088-4d3b-af2f-e8fc855e9ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.007612 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19590-0088-4d3b-af2f-e8fc855e9ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.008220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea19590-0088-4d3b-af2f-e8fc855e9ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.008427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.009054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.009368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.009640 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] PATH_INFO: `/v3/projects/e353e7c1c8b64117bb7a7c3a92d0d4b8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.010171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea19590-0088-4d3b-af2f-e8fc855e9ae6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.010819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e353e7c1c8b64117bb7a7c3a92d0d4b8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.013329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.034425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1265/2556] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:55 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:56.038490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19590-0088-4d3b-af2f-e8fc855e9ae6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.049220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19590-0088-4d3b-af2f-e8fc855e9ae6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.060208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:56.065094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e353e7c1c8b64117bb7a7c3a92d0d4b8', 'request_id': 'req-15889bd5-04ca-49df-accf-2b9570b30558'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:56.072100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e353e7c1c8b64117bb7a7c3a92d0d4b8', 'request_id': 'req-15889bd5-04ca-49df-accf-2b9570b30558'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:56.078893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e353e7c1c8b64117bb7a7c3a92d0d4b8', 'request_id': 'req-15889bd5-04ca-49df-accf-2b9570b30558'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:56.092192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] The token cache is being invalidate because project e353e7c1c8b64117bb7a7c3a92d0d4b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:56.092473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:56.093106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-15889bd5-04ca-49df-accf-2b9570b30558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:56.094897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1292/2557] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:55 2022] DELETE /identity/v3/projects/e353e7c1c8b64117bb7a7c3a92d0d4b8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:56.099656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33dc99-e338-4982-ad05-dc7abc7f4e25 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.099944 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33dc99-e338-4982-ad05-dc7abc7f4e25 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.100250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33dc99-e338-4982-ad05-dc7abc7f4e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.108472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.109251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.109648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.110016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.110762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.112495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.119980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9c33dc99-e338-4982-ad05-dc7abc7f4e25 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:56.121770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9c33dc99-e338-4982-ad05-dc7abc7f4e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.126838 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1266/2558] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:56.128166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.129694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.129863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1293/2559] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:56.142995 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6c2fd-4275-46c2-b7e4-c24844bc78c1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.148035 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334e956-1c2b-4b25-b54f-23bc01d1a744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.148677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334e956-1c2b-4b25-b54f-23bc01d1a744 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.148960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334e956-1c2b-4b25-b54f-23bc01d1a744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.149243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334e956-1c2b-4b25-b54f-23bc01d1a744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.149904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334e956-1c2b-4b25-b54f-23bc01d1a744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.151588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334e956-1c2b-4b25-b54f-23bc01d1a744 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.162890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1267/2560] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:56.169674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33dc99-e338-4982-ad05-dc7abc7f4e25 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.181478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33dc99-e338-4982-ad05-dc7abc7f4e25 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.187153 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aa1e8-57ac-48f7-be43-407cde97de53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.187822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa1e8-57ac-48f7-be43-407cde97de53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.188122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa1e8-57ac-48f7-be43-407cde97de53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.188392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa1e8-57ac-48f7-be43-407cde97de53 admin admin] PATH_INFO: `/v3/projects/04bd5514230b43d18044e894cf9bd535/users/ebef4f4d8a8f4d5d950b45d54a9c49c4/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.194483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084beda-677b-4273-ba4f-af755f4699d0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.194767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084beda-677b-4273-ba4f-af755f4699d0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.195041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084beda-677b-4273-ba4f-af755f4699d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.204466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1aa1e8-57ac-48f7-be43-407cde97de53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04bd5514230b43d18044e894cf9bd535, user_id=ebef4f4d8a8f4d5d950b45d54a9c49c4, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.207155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1aa1e8-57ac-48f7-be43-407cde97de53 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.215679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1084beda-677b-4273-ba4f-af755f4699d0 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:56.216879 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1084beda-677b-4273-ba4f-af755f4699d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.221191 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1294/2561] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:56 2022] PUT /identity/v3/projects/04bd5514230b43d18044e894cf9bd535/users/ebef4f4d8a8f4d5d950b45d54a9c49c4/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:56.231697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f6f2-703a-4829-86bd-f35a62c1aa21 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.231982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f6f2-703a-4829-86bd-f35a62c1aa21 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.232256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f6f2-703a-4829-86bd-f35a62c1aa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.261643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5cf2f6f2-703a-4829-86bd-f35a62c1aa21 None None] MFA Rules not processed for user `ebef4f4d8a8f4d5d950b45d54a9c49c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:56.263606 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5cf2f6f2-703a-4829-86bd-f35a62c1aa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.280199 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1268/2562] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:56.284409 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084beda-677b-4273-ba4f-af755f4699d0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.295945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084beda-677b-4273-ba4f-af755f4699d0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.300502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.301099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.301404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.301670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.302303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:56.302597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.304176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.316048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1269/2563] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:44:56.320524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.331634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1c04c-4112-44d9-868b-66edef7dca28 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.334576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1295/2564] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:56.337668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5adac-df56-470c-b2eb-557b7eddb141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.338619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5adac-df56-470c-b2eb-557b7eddb141 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.338893 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5adac-df56-470c-b2eb-557b7eddb141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.338998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2f6f2-703a-4829-86bd-f35a62c1aa21 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.339158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5adac-df56-470c-b2eb-557b7eddb141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.339796 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5adac-df56-470c-b2eb-557b7eddb141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.341803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5adac-df56-470c-b2eb-557b7eddb141 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.349368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2f6f2-703a-4829-86bd-f35a62c1aa21 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.354031 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ad344-3cce-44bb-9d24-93be3edf60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.354628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ad344-3cce-44bb-9d24-93be3edf60cd admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.354905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ad344-3cce-44bb-9d24-93be3edf60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.355208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ad344-3cce-44bb-9d24-93be3edf60cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.355829 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ad344-3cce-44bb-9d24-93be3edf60cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.357790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ad344-3cce-44bb-9d24-93be3edf60cd admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.370150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1270/2565] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:56.374668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5adac-df56-470c-b2eb-557b7eddb141 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.385245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:56.389329 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1296/2566] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:56.391262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ad344-3cce-44bb-9d24-93be3edf60cd admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.439498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5adac-df56-470c-b2eb-557b7eddb141 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.455193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ad344-3cce-44bb-9d24-93be3edf60cd admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.460081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e6cc-34e5-4bdf-afc0-93a4fd8d3b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.460680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e6cc-34e5-4bdf-afc0-93a4fd8d3b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.460958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e6cc-34e5-4bdf-afc0-93a4fd8d3b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.461247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e6cc-34e5-4bdf-afc0-93a4fd8d3b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.461922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e6cc-34e5-4bdf-afc0-93a4fd8d3b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.463836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e6cc-34e5-4bdf-afc0-93a4fd8d3b1a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.479903 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:56.487389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1297/2567] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:56.491759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e6cc-34e5-4bdf-afc0-93a4fd8d3b1a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.502986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e6cc-34e5-4bdf-afc0-93a4fd8d3b1a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.503609 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2cae8-0e31-4c32-a94a-8bfe04c1186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.504226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2cae8-0e31-4c32-a94a-8bfe04c1186a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.504502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2cae8-0e31-4c32-a94a-8bfe04c1186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.504767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2cae8-0e31-4c32-a94a-8bfe04c1186a admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.505431 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2cae8-0e31-4c32-a94a-8bfe04c1186a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.507382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2cae8-0e31-4c32-a94a-8bfe04c1186a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.507875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ce961-e1ef-4cbd-8cfa-6b9d0aa19b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.508468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ce961-e1ef-4cbd-8cfa-6b9d0aa19b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.508743 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ce961-e1ef-4cbd-8cfa-6b9d0aa19b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.509014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ce961-e1ef-4cbd-8cfa-6b9d0aa19b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.509683 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ce961-e1ef-4cbd-8cfa-6b9d0aa19b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.511265 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ce961-e1ef-4cbd-8cfa-6b9d0aa19b98 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.525140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1298/2568] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:56.526108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ce961-e1ef-4cbd-8cfa-6b9d0aa19b98 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.532747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1271/2569] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:56.534118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2cae8-0e31-4c32-a94a-8bfe04c1186a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.536239 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ce961-e1ef-4cbd-8cfa-6b9d0aa19b98 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.540986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390ef10-1071-404a-94e6-c86142fdfb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.541664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ef10-1071-404a-94e6-c86142fdfb1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.541943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ef10-1071-404a-94e6-c86142fdfb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.542221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ef10-1071-404a-94e6-c86142fdfb1b admin admin] PATH_INFO: `/v3/projects/da6cbd2ba9054ab6bc65407202b83ba7/users/fe37af36d422430baa4df99d2ed99df5/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.544296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2cae8-0e31-4c32-a94a-8bfe04c1186a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.549622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334832c-4a3f-4c1a-b599-8800225ebc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.550149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334832c-4a3f-4c1a-b599-8800225ebc3a admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.550523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334832c-4a3f-4c1a-b599-8800225ebc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.550795 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334832c-4a3f-4c1a-b599-8800225ebc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.551454 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334832c-4a3f-4c1a-b599-8800225ebc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.553032 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334832c-4a3f-4c1a-b599-8800225ebc3a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.560736 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390ef10-1071-404a-94e6-c86142fdfb1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6cbd2ba9054ab6bc65407202b83ba7, user_id=fe37af36d422430baa4df99d2ed99df5, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.563454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390ef10-1071-404a-94e6-c86142fdfb1b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.564530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1272/2570] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:56.568999 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334832c-4a3f-4c1a-b599-8800225ebc3a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.577811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1299/2571] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:56 2022] PUT /identity/v3/projects/da6cbd2ba9054ab6bc65407202b83ba7/users/fe37af36d422430baa4df99d2ed99df5/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:56.579188 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334832c-4a3f-4c1a-b599-8800225ebc3a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.581725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390ef10-1071-404a-94e6-c86142fdfb1b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.640401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72eaf0-5388-4757-8def-f81dd8739a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.641017 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72eaf0-5388-4757-8def-f81dd8739a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.641321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72eaf0-5388-4757-8def-f81dd8739a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.641591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72eaf0-5388-4757-8def-f81dd8739a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.642229 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72eaf0-5388-4757-8def-f81dd8739a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.644044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72eaf0-5388-4757-8def-f81dd8739a3b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.650418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390ef10-1071-404a-94e6-c86142fdfb1b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.655414 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1273/2572] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:56.659376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72eaf0-5388-4757-8def-f81dd8739a3b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.662464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffa3db-169f-42fe-a6dd-0ed898ae7ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.663085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffa3db-169f-42fe-a6dd-0ed898ae7ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.663362 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffa3db-169f-42fe-a6dd-0ed898ae7ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.663685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffa3db-169f-42fe-a6dd-0ed898ae7ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.664283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffa3db-169f-42fe-a6dd-0ed898ae7ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.666016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffa3db-169f-42fe-a6dd-0ed898ae7ed0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.670071 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72eaf0-5388-4757-8def-f81dd8739a3b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.674853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156cd5a-f01e-4797-aaa8-18db108592c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.675507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156cd5a-f01e-4797-aaa8-18db108592c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.675786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156cd5a-f01e-4797-aaa8-18db108592c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.676055 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156cd5a-f01e-4797-aaa8-18db108592c4 admin admin] PATH_INFO: `/v3/projects/2235b02be77e4eb29499331aa2535421/users/9b6ef1675a574d92be3902ee97289421/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.680912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1300/2573] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:56.682019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffa3db-169f-42fe-a6dd-0ed898ae7ed0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.695708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffa3db-169f-42fe-a6dd-0ed898ae7ed0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.698070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156cd5a-f01e-4797-aaa8-18db108592c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2235b02be77e4eb29499331aa2535421, user_id=9b6ef1675a574d92be3902ee97289421, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.701067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312f413-0976-420d-a41b-423d94978862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.701785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312f413-0976-420d-a41b-423d94978862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.702101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312f413-0976-420d-a41b-423d94978862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.702321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312f413-0976-420d-a41b-423d94978862 admin admin] PATH_INFO: `/v3/projects/da6cbd2ba9054ab6bc65407202b83ba7/users/fe37af36d422430baa4df99d2ed99df5/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.703296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156cd5a-f01e-4797-aaa8-18db108592c4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.704234 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312f413-0976-420d-a41b-423d94978862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6cbd2ba9054ab6bc65407202b83ba7, user_id=fe37af36d422430baa4df99d2ed99df5, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.706946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312f413-0976-420d-a41b-423d94978862 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.718225 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1301/2574] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:56 2022] PUT /identity/v3/projects/da6cbd2ba9054ab6bc65407202b83ba7/users/fe37af36d422430baa4df99d2ed99df5/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:56.728026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1274/2575] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:56 2022] PUT /identity/v3/projects/2235b02be77e4eb29499331aa2535421/users/9b6ef1675a574d92be3902ee97289421/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:44:56.731905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda9ce-a74a-49db-83a7-108c17b9f472 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.731905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda9ce-a74a-49db-83a7-108c17b9f472 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.733584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b6bd6-ee2e-4684-9688-e92ea3435c56 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.734987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda9ce-a74a-49db-83a7-108c17b9f472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.735324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b6bd6-ee2e-4684-9688-e92ea3435c56 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.735324 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b6bd6-ee2e-4684-9688-e92ea3435c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.748750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:56.762039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1ceda9ce-a74a-49db-83a7-108c17b9f472 None None] MFA Rules not processed for user `9b6ef1675a574d92be3902ee97289421`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:56.763430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1ceda9ce-a74a-49db-83a7-108c17b9f472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.763780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-796b6bd6-ee2e-4684-9688-e92ea3435c56 None None] MFA Rules not processed for user `fe37af36d422430baa4df99d2ed99df5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:56.765020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-796b6bd6-ee2e-4684-9688-e92ea3435c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.841419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1302/2576] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:56.844215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda9ce-a74a-49db-83a7-108c17b9f472 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.845907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1ceda9ce-a74a-49db-83a7-108c17b9f472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.847339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1275/2577] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:56.852953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b6bd6-ee2e-4684-9688-e92ea3435c56 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.858132 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-796b6bd6-ee2e-4684-9688-e92ea3435c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.888257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:56.909295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda9ce-a74a-49db-83a7-108c17b9f472 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.913121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.913866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.914140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.914410 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.916323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.916900 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b6bd6-ee2e-4684-9688-e92ea3435c56 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.921373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.921981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.922259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.922527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.930825 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.932397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.941886 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1276/2578] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:56.944638 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:56.945971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:56.958568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d15322-f563-43bf-83ed-f50f41e053b7 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:56.962847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413a459-2982-4216-92e2-ce5e9b03e19a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:56.963456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413a459-2982-4216-92e2-ce5e9b03e19a service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:56.963736 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413a459-2982-4216-92e2-ce5e9b03e19a service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:56.964007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413a459-2982-4216-92e2-ce5e9b03e19a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:56.978702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413a459-2982-4216-92e2-ce5e9b03e19a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.981137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413a459-2982-4216-92e2-ce5e9b03e19a service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.984071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:56.988953 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:56.991687 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1277/2579] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:56.999176 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1303/2580] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:57.095147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413a459-2982-4216-92e2-ce5e9b03e19a service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:57.109069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413a459-2982-4216-92e2-ce5e9b03e19a service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:57.113332 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fcb6b-d6ce-452a-a33c-fc484d9a665d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:57.114129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fcb6b-d6ce-452a-a33c-fc484d9a665d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:57.114226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fcb6b-d6ce-452a-a33c-fc484d9a665d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:57.114461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fcb6b-d6ce-452a-a33c-fc484d9a665d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:57.189364 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fcb6b-d6ce-452a-a33c-fc484d9a665d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:57.190783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fcb6b-d6ce-452a-a33c-fc484d9a665d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:57.202235 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1278/2581] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:57.632416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:57.641411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e4a28-4d37-4772-a89b-f98e2b653107 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:57.647156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:57.648004 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:57.648356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:57.648730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] PATH_INFO: `/v3/users/ebef4f4d8a8f4d5d950b45d54a9c49c4` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:57.650439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebef4f4d8a8f4d5d950b45d54a9c49c4)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:57.652816 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:57.708301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebef4f4d8a8f4d5d950b45d54a9c49c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:57.716637 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebef4f4d8a8f4d5d950b45d54a9c49c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:57.717569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebef4f4d8a8f4d5d950b45d54a9c49c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:57.730257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebef4f4d8a8f4d5d950b45d54a9c49c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:57.732330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebef4f4d8a8f4d5d950b45d54a9c49c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:57.743863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1304/2582] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:57 2022] DELETE /identity/v3/users/ebef4f4d8a8f4d5d950b45d54a9c49c4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:57.749137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394770a3-8c3f-4a58-a07c-8f88eaf8d2dc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:57.749626 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394770a3-8c3f-4a58-a07c-8f88eaf8d2dc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:57.749738 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394770a3-8c3f-4a58-a07c-8f88eaf8d2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:57.768484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-394770a3-8c3f-4a58-a07c-8f88eaf8d2dc None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:57.769717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-394770a3-8c3f-4a58-a07c-8f88eaf8d2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:57.783270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:57.831397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1279/2583] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:57.842222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:57.843715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:57.909150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c76a5c-e038-4dae-84e8-10e56689e959 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:57.912760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89fd3d-ad6b-49b7-85fb-6befa2836db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:57.913465 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89fd3d-ad6b-49b7-85fb-6befa2836db1 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:57.913739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89fd3d-ad6b-49b7-85fb-6befa2836db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:57.914003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89fd3d-ad6b-49b7-85fb-6befa2836db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:57.924059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89fd3d-ad6b-49b7-85fb-6befa2836db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:57.925632 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89fd3d-ad6b-49b7-85fb-6befa2836db1 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:57.932416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:57.937980 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1305/2584] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:44:58.392419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394770a3-8c3f-4a58-a07c-8f88eaf8d2dc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:58.397054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-394770a3-8c3f-4a58-a07c-8f88eaf8d2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:58.411241 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394770a3-8c3f-4a58-a07c-8f88eaf8d2dc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:58.416832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:58.417503 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:58.418642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:58.418913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] PATH_INFO: `/v3/projects/04bd5514230b43d18044e894cf9bd535` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:58.420449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=04bd5514230b43d18044e894cf9bd535)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:58.423137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:58.490622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04bd5514230b43d18044e894cf9bd535', 'request_id': 'req-7f158355-a800-4881-8a73-21b8469585cf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.497294 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04bd5514230b43d18044e894cf9bd535', 'request_id': 'req-7f158355-a800-4881-8a73-21b8469585cf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.504263 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04bd5514230b43d18044e894cf9bd535', 'request_id': 'req-7f158355-a800-4881-8a73-21b8469585cf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.524869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] The token cache is being invalidate because project 04bd5514230b43d18044e894cf9bd535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:58.525276 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.526003 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.529317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1280/2585] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:58 2022] DELETE /identity/v3/projects/04bd5514230b43d18044e894cf9bd535 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:58.534011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89fd3d-ad6b-49b7-85fb-6befa2836db1 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:58.547646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89fd3d-ad6b-49b7-85fb-6befa2836db1 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:58.616756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:58.624068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:58.624614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:58.625243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] PATH_INFO: `/v3/users/fe37af36d422430baa4df99d2ed99df5` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:58.626783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe37af36d422430baa4df99d2ed99df5)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:58.629166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:58.738218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe37af36d422430baa4df99d2ed99df5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.758718 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe37af36d422430baa4df99d2ed99df5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.759427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe37af36d422430baa4df99d2ed99df5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.776909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe37af36d422430baa4df99d2ed99df5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.778461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4249f4f8-bc20-4fac-8ff6-3feebaf95770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe37af36d422430baa4df99d2ed99df5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:58.810602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1306/2586] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:44:58 2022] DELETE /identity/v3/users/fe37af36d422430baa4df99d2ed99df5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:59.155004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:59.156620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:59.173131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f158355-a800-4881-8a73-21b8469585cf admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:59.244493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:59.245182 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.245512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.246868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] PATH_INFO: `/v3/projects/da6cbd2ba9054ab6bc65407202b83ba7` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.247707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6cbd2ba9054ab6bc65407202b83ba7)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:59.250073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:59.309859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:59.314929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6cbd2ba9054ab6bc65407202b83ba7', 'request_id': 'req-a862025f-940f-40a0-a5a7-5bb7c5468a65'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:59.325444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6cbd2ba9054ab6bc65407202b83ba7', 'request_id': 'req-a862025f-940f-40a0-a5a7-5bb7c5468a65'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:59.332728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6cbd2ba9054ab6bc65407202b83ba7', 'request_id': 'req-a862025f-940f-40a0-a5a7-5bb7c5468a65'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:59.348179 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] The token cache is being invalidate because project da6cbd2ba9054ab6bc65407202b83ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:44:59.348554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:59.349256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a862025f-940f-40a0-a5a7-5bb7c5468a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:44:59.351349 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1281/2587] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:44:59 2022] DELETE /identity/v3/projects/da6cbd2ba9054ab6bc65407202b83ba7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:59.356150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384342f4-2ec2-4f33-b83d-5acfd32832db None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.356585 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384342f4-2ec2-4f33-b83d-5acfd32832db None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.356910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384342f4-2ec2-4f33-b83d-5acfd32832db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.380387 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-384342f4-2ec2-4f33-b83d-5acfd32832db None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:59.381102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-384342f4-2ec2-4f33-b83d-5acfd32832db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:59.454577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1307/2588] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:44:59.462317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347d6f4-5bd0-4fd4-ac5e-2eccbd2ac3b9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.462594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347d6f4-5bd0-4fd4-ac5e-2eccbd2ac3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.462862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347d6f4-5bd0-4fd4-ac5e-2eccbd2ac3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.482913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9347d6f4-5bd0-4fd4-ac5e-2eccbd2ac3b9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:59.484127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9347d6f4-5bd0-4fd4-ac5e-2eccbd2ac3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:44:59.492382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1282/2589] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:44:59.497870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384342f4-2ec2-4f33-b83d-5acfd32832db None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:59.507565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384342f4-2ec2-4f33-b83d-5acfd32832db None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:59.512363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:59.512964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.513291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.513566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.514239 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:44:59.514685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:59.516323 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:59.531649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1308/2590] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:44:59.536275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347d6f4-5bd0-4fd4-ac5e-2eccbd2ac3b9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:59.546945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347d6f4-5bd0-4fd4-ac5e-2eccbd2ac3b9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:59.551867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d06cdb-93ac-47cd-b3bd-ee28011cca4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:59.552479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d06cdb-93ac-47cd-b3bd-ee28011cca4c admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.552757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d06cdb-93ac-47cd-b3bd-ee28011cca4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.553033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d06cdb-93ac-47cd-b3bd-ee28011cca4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.553729 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d06cdb-93ac-47cd-b3bd-ee28011cca4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:59.555711 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d06cdb-93ac-47cd-b3bd-ee28011cca4c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:59.583441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1283/2591] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:44:59 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:44:59.587614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:59.597926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17441-fda7-4864-9183-3bc6ae4f2205 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:59.601597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:44:59.658888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f8b3-b75d-4cef-a2f0-8fa5aa29cc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:59.659427 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f8b3-b75d-4cef-a2f0-8fa5aa29cc04 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.659706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f8b3-b75d-4cef-a2f0-8fa5aa29cc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.659975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f8b3-b75d-4cef-a2f0-8fa5aa29cc04 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.660605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89f8b3-b75d-4cef-a2f0-8fa5aa29cc04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:59.662640 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89f8b3-b75d-4cef-a2f0-8fa5aa29cc04 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:59.688661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1309/2592] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:44:59 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:44:59.689916 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d06cdb-93ac-47cd-b3bd-ee28011cca4c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:59.700536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d06cdb-93ac-47cd-b3bd-ee28011cca4c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:59.705220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe049f-4618-4edf-a65c-dee1827aaa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:59.706491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe049f-4618-4edf-a65c-dee1827aaa2d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.706775 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe049f-4618-4edf-a65c-dee1827aaa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.707079 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe049f-4618-4edf-a65c-dee1827aaa2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.707751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe049f-4618-4edf-a65c-dee1827aaa2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:59.709367 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe049f-4618-4edf-a65c-dee1827aaa2d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:59.729420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1284/2593] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:44:59.733724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f8b3-b75d-4cef-a2f0-8fa5aa29cc04 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:59.759373 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f8b3-b75d-4cef-a2f0-8fa5aa29cc04 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:59.771349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71641b6-d07d-4d78-92ef-f327e9b20a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:59.772104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71641b6-d07d-4d78-92ef-f327e9b20a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.772450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71641b6-d07d-4d78-92ef-f327e9b20a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.772791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71641b6-d07d-4d78-92ef-f327e9b20a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.777995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71641b6-d07d-4d78-92ef-f327e9b20a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:59.780005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71641b6-d07d-4d78-92ef-f327e9b20a92 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:59.801320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1310/2594] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:44:59.805573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe049f-4618-4edf-a65c-dee1827aaa2d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:44:59.816045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe049f-4618-4edf-a65c-dee1827aaa2d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:44:59.821327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750effce-68f9-406e-97a5-057d9b850473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:44:59.822070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750effce-68f9-406e-97a5-057d9b850473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.822427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750effce-68f9-406e-97a5-057d9b850473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.822980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750effce-68f9-406e-97a5-057d9b850473 admin admin] PATH_INFO: `/v3/projects/2c8155a10bc44935988111c19ccf70ad/users/2c58a4950da34c189b7a1b82ebc9289d/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.844334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750effce-68f9-406e-97a5-057d9b850473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8155a10bc44935988111c19ccf70ad, user_id=2c58a4950da34c189b7a1b82ebc9289d, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:44:59.847165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750effce-68f9-406e-97a5-057d9b850473 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:44:59.877266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1285/2595] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:44:59 2022] PUT /identity/v3/projects/2c8155a10bc44935988111c19ccf70ad/users/2c58a4950da34c189b7a1b82ebc9289d/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:44:59.939332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288748-3d24-406f-aaad-72c5d3a30d37 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:44:59.939786 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288748-3d24-406f-aaad-72c5d3a30d37 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:44:59.940105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288748-3d24-406f-aaad-72c5d3a30d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:44:59.976853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-89288748-3d24-406f-aaad-72c5d3a30d37 None None] MFA Rules not processed for user `2c58a4950da34c189b7a1b82ebc9289d`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:44:59.978444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-89288748-3d24-406f-aaad-72c5d3a30d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:00.067214 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1311/2596] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:00.071872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750effce-68f9-406e-97a5-057d9b850473 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.171591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750effce-68f9-406e-97a5-057d9b850473 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:00.178459 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43380-cf0f-4e3e-87ee-edcf3ab55a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:00.179089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43380-cf0f-4e3e-87ee-edcf3ab55a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:00.179368 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43380-cf0f-4e3e-87ee-edcf3ab55a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:00.189735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43380-cf0f-4e3e-87ee-edcf3ab55a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:00.189735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb43380-cf0f-4e3e-87ee-edcf3ab55a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:00.189735 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb43380-cf0f-4e3e-87ee-edcf3ab55a36 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:00.275292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1286/2597] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:00 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:00.285819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89288748-3d24-406f-aaad-72c5d3a30d37 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.306636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89288748-3d24-406f-aaad-72c5d3a30d37 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:00.375079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbb62e-75cf-4825-8069-a2b1bd71d4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:00.375740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb62e-75cf-4825-8069-a2b1bd71d4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:00.376017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb62e-75cf-4825-8069-a2b1bd71d4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:00.376289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb62e-75cf-4825-8069-a2b1bd71d4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:00.376923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbb62e-75cf-4825-8069-a2b1bd71d4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:00.378997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbb62e-75cf-4825-8069-a2b1bd71d4c6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:00.412658 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1312/2598] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:00 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:00.416709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43380-cf0f-4e3e-87ee-edcf3ab55a36 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.428434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43380-cf0f-4e3e-87ee-edcf3ab55a36 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:00.434403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d634cbb-59b6-4970-bc97-29db01a7a81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:00.434772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d634cbb-59b6-4970-bc97-29db01a7a81b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:00.435096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d634cbb-59b6-4970-bc97-29db01a7a81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:00.435458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d634cbb-59b6-4970-bc97-29db01a7a81b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:00.436212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d634cbb-59b6-4970-bc97-29db01a7a81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:00.437994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d634cbb-59b6-4970-bc97-29db01a7a81b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:00.453308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1287/2599] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:45:00.456201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbb62e-75cf-4825-8069-a2b1bd71d4c6 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.463777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:00.469691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbb62e-75cf-4825-8069-a2b1bd71d4c6 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:00.475694 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:00.476396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:00.477059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:00.478537 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:00.479216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:00.480913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:00.497288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1313/2600] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:00.499501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d634cbb-59b6-4970-bc97-29db01a7a81b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.512603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d634cbb-59b6-4970-bc97-29db01a7a81b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:00.519410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642e6a0-d7ee-483b-9998-32f74b6126ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:00.520579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642e6a0-d7ee-483b-9998-32f74b6126ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:00.520867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642e6a0-d7ee-483b-9998-32f74b6126ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:00.521397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642e6a0-d7ee-483b-9998-32f74b6126ed admin admin] PATH_INFO: `/v3/projects/0c675f2d1be44bdc8923caac3c54d746/users/a7f16430c5a342c78e0e04a53aaf34f4/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:00.536724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.538600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:00.559550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21720af7-48d5-4af9-8695-e71505636a7a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:00.561889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642e6a0-d7ee-483b-9998-32f74b6126ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c675f2d1be44bdc8923caac3c54d746, user_id=a7f16430c5a342c78e0e04a53aaf34f4, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:00.565852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642e6a0-d7ee-483b-9998-32f74b6126ed admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:00.575436 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:00.576398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:00.576708 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:00.577028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] PATH_INFO: `/v3/users/9cecae4323404bd29fe8c05a704de273` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:00.579102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cecae4323404bd29fe8c05a704de273)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:00.581715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:00.588042 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1288/2601] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:00 2022] PUT /identity/v3/projects/0c675f2d1be44bdc8923caac3c54d746/users/a7f16430c5a342c78e0e04a53aaf34f4/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:00.598741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80791bca-50a4-4de8-9e5c-a277eee78dde None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:00.599034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80791bca-50a4-4de8-9e5c-a277eee78dde None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:00.599434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80791bca-50a4-4de8-9e5c-a277eee78dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:00.650008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-80791bca-50a4-4de8-9e5c-a277eee78dde None None] MFA Rules not processed for user `a7f16430c5a342c78e0e04a53aaf34f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:00.651250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-80791bca-50a4-4de8-9e5c-a277eee78dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:00.661070 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:00.675283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cecae4323404bd29fe8c05a704de273'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:00.686474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cecae4323404bd29fe8c05a704de273'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:00.687371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cecae4323404bd29fe8c05a704de273'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:00.701541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cecae4323404bd29fe8c05a704de273'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:00.702228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cecae4323404bd29fe8c05a704de273'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:00.714936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1314/2602] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:00 2022] DELETE /identity/v3/users/9cecae4323404bd29fe8c05a704de273 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:00.736987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1289/2603] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:00.741674 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.743066 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:00.823984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bb047-7ce3-4057-bcd5-00fdbd4a4c50 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:00.830144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:00.830755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:00.831025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:00.831294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:00.831925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:00.833931 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:00.872316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1315/2604] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:00 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:00.873463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80791bca-50a4-4de8-9e5c-a277eee78dde None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.915001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:00.916968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:00.957746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80791bca-50a4-4de8-9e5c-a277eee78dde None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:00.981839 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:01.001844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26b4c6-5ede-4baf-9774-4862b4bf71fa admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.006948 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.007659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.008010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.008350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] PATH_INFO: `/v3/projects/b6cfb6539b69493ebef06cf7176b1935` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.009753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6cfb6539b69493ebef06cf7176b1935)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.011682 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.018092 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d02dcd-3b6d-4413-8f0d-b5108bb3773a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.018935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d02dcd-3b6d-4413-8f0d-b5108bb3773a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.019412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d02dcd-3b6d-4413-8f0d-b5108bb3773a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.019968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d02dcd-3b6d-4413-8f0d-b5108bb3773a admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.020936 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d02dcd-3b6d-4413-8f0d-b5108bb3773a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.022874 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d02dcd-3b6d-4413-8f0d-b5108bb3773a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.044938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1290/2605] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:00 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:01.049020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d02dcd-3b6d-4413-8f0d-b5108bb3773a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.059148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d02dcd-3b6d-4413-8f0d-b5108bb3773a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.066529 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6cfb6539b69493ebef06cf7176b1935', 'request_id': 'req-bec8a34b-0462-4f95-8a47-52c89f934feb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:01.072209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6cfb6539b69493ebef06cf7176b1935', 'request_id': 'req-bec8a34b-0462-4f95-8a47-52c89f934feb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:01.079034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6cfb6539b69493ebef06cf7176b1935', 'request_id': 'req-bec8a34b-0462-4f95-8a47-52c89f934feb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:01.091646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] The token cache is being invalidate because project b6cfb6539b69493ebef06cf7176b1935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:01.091932 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:01.092575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bec8a34b-0462-4f95-8a47-52c89f934feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:01.097645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1316/2606] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:00 2022] DELETE /identity/v3/projects/b6cfb6539b69493ebef06cf7176b1935 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:45:01.100579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490998-b6b1-405d-8b35-cfacc4725934 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.100864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490998-b6b1-405d-8b35-cfacc4725934 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.101165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490998-b6b1-405d-8b35-cfacc4725934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.105863 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:01.122100 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308fa44-40d4-4229-8f28-c0e69074ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.122933 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308fa44-40d4-4229-8f28-c0e69074ec53 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.123207 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308fa44-40d4-4229-8f28-c0e69074ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.123608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308fa44-40d4-4229-8f28-c0e69074ec53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.123966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-aa490998-b6b1-405d-8b35-cfacc4725934 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:01.124248 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308fa44-40d4-4229-8f28-c0e69074ec53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.125178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aa490998-b6b1-405d-8b35-cfacc4725934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:01.125858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308fa44-40d4-4229-8f28-c0e69074ec53 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.142345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1317/2607] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:01.142558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1291/2608] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:01.142663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa490998-b6b1-405d-8b35-cfacc4725934 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.142663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aa490998-b6b1-405d-8b35-cfacc4725934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:01.143832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25217a3c-4152-40d7-b0c6-540d453d762e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.144107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25217a3c-4152-40d7-b0c6-540d453d762e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.144375 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25217a3c-4152-40d7-b0c6-540d453d762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.154161 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa490998-b6b1-405d-8b35-cfacc4725934 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.158836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e215c9-a0bb-4c37-a931-db0f6b99d73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.159478 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e215c9-a0bb-4c37-a931-db0f6b99d73c admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.159753 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e215c9-a0bb-4c37-a931-db0f6b99d73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.160022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e215c9-a0bb-4c37-a931-db0f6b99d73c admin admin] PATH_INFO: `/v3/projects/205ff48977a041d281cd561fea437957/users/1b4854f5c7e747c5b887b2c0ff60b2d3/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.165658 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-25217a3c-4152-40d7-b0c6-540d453d762e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:01.166870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-25217a3c-4152-40d7-b0c6-540d453d762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:01.176127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1292/2609] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:01.176704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e215c9-a0bb-4c37-a931-db0f6b99d73c admin admin] RBAC: Authorizing `identity:create_grant(project_id=205ff48977a041d281cd561fea437957, user_id=1b4854f5c7e747c5b887b2c0ff60b2d3, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.178808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e215c9-a0bb-4c37-a931-db0f6b99d73c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.180509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25217a3c-4152-40d7-b0c6-540d453d762e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.192603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25217a3c-4152-40d7-b0c6-540d453d762e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.195646 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1318/2610] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:01 2022] PUT /identity/v3/projects/205ff48977a041d281cd561fea437957/users/1b4854f5c7e747c5b887b2c0ff60b2d3/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:45:01.199445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e215c9-a0bb-4c37-a931-db0f6b99d73c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.256752 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.257418 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.257699 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.258020 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.258599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:45:01.258897 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.260480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.273704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e215c9-a0bb-4c37-a931-db0f6b99d73c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.277911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1293/2611] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:45:01.279348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.280166 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b573f-c741-4703-8c92-69a2e56bb200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.280776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b573f-c741-4703-8c92-69a2e56bb200 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.281047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b573f-c741-4703-8c92-69a2e56bb200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.281365 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b573f-c741-4703-8c92-69a2e56bb200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.282140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b573f-c741-4703-8c92-69a2e56bb200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.283990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b573f-c741-4703-8c92-69a2e56bb200 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.293397 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba1db1-5d86-462b-8616-dada401c02e8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.295092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1319/2612] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:45:01.299049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b573f-c741-4703-8c92-69a2e56bb200 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.301565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f5889-107c-4aad-b3e8-291d0486453b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.302359 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5889-107c-4aad-b3e8-291d0486453b admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.302696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5889-107c-4aad-b3e8-291d0486453b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.303027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5889-107c-4aad-b3e8-291d0486453b admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.303706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f5889-107c-4aad-b3e8-291d0486453b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.305865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f5889-107c-4aad-b3e8-291d0486453b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.309162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b573f-c741-4703-8c92-69a2e56bb200 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.314532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b3f61-216e-4388-9690-ad4d015d6789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.315169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b3f61-216e-4388-9690-ad4d015d6789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.315447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b3f61-216e-4388-9690-ad4d015d6789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.315723 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b3f61-216e-4388-9690-ad4d015d6789 admin admin] PATH_INFO: `/v3/projects/205ff48977a041d281cd561fea437957/users/1b4854f5c7e747c5b887b2c0ff60b2d3/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.318117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b3f61-216e-4388-9690-ad4d015d6789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=205ff48977a041d281cd561fea437957, user_id=1b4854f5c7e747c5b887b2c0ff60b2d3, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.321382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b3f61-216e-4388-9690-ad4d015d6789 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.333602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1320/2613] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:01 2022] PUT /identity/v3/projects/205ff48977a041d281cd561fea437957/users/1b4854f5c7e747c5b887b2c0ff60b2d3/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:45:01.340436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1294/2614] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:01 2022] POST /identity/v3/projects => generated 355 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:01.343977 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608ea5b-ed96-4149-9c43-3ee6fd4f1b24 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.344254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608ea5b-ed96-4149-9c43-3ee6fd4f1b24 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.344382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f5889-107c-4aad-b3e8-291d0486453b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.344642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608ea5b-ed96-4149-9c43-3ee6fd4f1b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.375574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4608ea5b-ed96-4149-9c43-3ee6fd4f1b24 None None] MFA Rules not processed for user `1b4854f5c7e747c5b887b2c0ff60b2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:01.376793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4608ea5b-ed96-4149-9c43-3ee6fd4f1b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:01.399256 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:01.432696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f5889-107c-4aad-b3e8-291d0486453b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.437696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.438433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.438720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.438989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.439598 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.441741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.459295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1321/2615] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:01.465200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1295/2616] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:01 2022] POST /identity/v3/users => generated 278 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:01.469109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608ea5b-ed96-4149-9c43-3ee6fd4f1b24 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.474266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.476091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:01.479773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608ea5b-ed96-4149-9c43-3ee6fd4f1b24 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.485545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344483e-47ef-417d-9e95-98a4063aef7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.486140 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344483e-47ef-417d-9e95-98a4063aef7e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.486446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344483e-47ef-417d-9e95-98a4063aef7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.486711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344483e-47ef-417d-9e95-98a4063aef7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.487341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344483e-47ef-417d-9e95-98a4063aef7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.489267 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344483e-47ef-417d-9e95-98a4063aef7e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.499268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1322/2617] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:45:01.503104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344483e-47ef-417d-9e95-98a4063aef7e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.512895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344483e-47ef-417d-9e95-98a4063aef7e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.517499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f356-d53d-44c8-84ea-ff2a6374325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.518144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f356-d53d-44c8-84ea-ff2a6374325c admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.518419 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f356-d53d-44c8-84ea-ff2a6374325c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.518690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f356-d53d-44c8-84ea-ff2a6374325c admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.519351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f356-d53d-44c8-84ea-ff2a6374325c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.520142 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:01.520936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f356-d53d-44c8-84ea-ff2a6374325c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.535123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1323/2618] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:45:01.536255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f356-d53d-44c8-84ea-ff2a6374325c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.537523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3cfc5-b4e4-4928-ac8b-28fcd9256bdc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.541148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.541785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.542577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.542577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.544666 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:01.546399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f356-d53d-44c8-84ea-ff2a6374325c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.551066 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f1ce4-c11a-48c0-a6fe-706cc7f64f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.557417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1ce4-c11a-48c0-a6fe-706cc7f64f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.557748 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1ce4-c11a-48c0-a6fe-706cc7f64f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.558037 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1ce4-c11a-48c0-a6fe-706cc7f64f80 admin admin] PATH_INFO: `/v3/projects/29af2ad658b040869214a908ed82b517/users/fc65bfeeff464616886cddb8713a5354/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.572842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f1ce4-c11a-48c0-a6fe-706cc7f64f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29af2ad658b040869214a908ed82b517, user_id=fc65bfeeff464616886cddb8713a5354, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.575557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f1ce4-c11a-48c0-a6fe-706cc7f64f80 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.589213 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1324/2619] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:01 2022] PUT /identity/v3/projects/29af2ad658b040869214a908ed82b517/users/fc65bfeeff464616886cddb8713a5354/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:45:01.600046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235913e-6004-4548-8fa9-14e0dbd747b7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.600327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235913e-6004-4548-8fa9-14e0dbd747b7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.600594 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235913e-6004-4548-8fa9-14e0dbd747b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.606264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.607703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.617685 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1296/2620] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:45:01.630612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1235913e-6004-4548-8fa9-14e0dbd747b7 None None] MFA Rules not processed for user `fc65bfeeff464616886cddb8713a5354`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:01.631840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1235913e-6004-4548-8fa9-14e0dbd747b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:01.696409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1325/2621] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:01.701618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.712813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7163d-78d4-41ce-bf02-fb07168ac41c service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.778048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b23a5d-0e44-4952-b218-59f1211950b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.778832 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b23a5d-0e44-4952-b218-59f1211950b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.779086 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b23a5d-0e44-4952-b218-59f1211950b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.779413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b23a5d-0e44-4952-b218-59f1211950b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.780135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b23a5d-0e44-4952-b218-59f1211950b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.782273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b23a5d-0e44-4952-b218-59f1211950b7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.808892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1297/2622] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:01 2022] POST /identity/v3/projects => generated 355 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:01.813075 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235913e-6004-4548-8fa9-14e0dbd747b7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.824788 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235913e-6004-4548-8fa9-14e0dbd747b7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.914936 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3cf2-8a9f-4647-a890-b583e4130407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.915547 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3cf2-8a9f-4647-a890-b583e4130407 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.915822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3cf2-8a9f-4647-a890-b583e4130407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.916088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3cf2-8a9f-4647-a890-b583e4130407 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.916823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc3cf2-8a9f-4647-a890-b583e4130407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.920345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc3cf2-8a9f-4647-a890-b583e4130407 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.930159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:01.945248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1326/2623] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:01 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:01.949593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b23a5d-0e44-4952-b218-59f1211950b7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.961185 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b23a5d-0e44-4952-b218-59f1211950b7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.966820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c98f7-6faa-41de-8428-7906c0059b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:01.967433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c98f7-6faa-41de-8428-7906c0059b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:01.967708 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c98f7-6faa-41de-8428-7906c0059b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:01.968000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c98f7-6faa-41de-8428-7906c0059b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:01.968642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c98f7-6faa-41de-8428-7906c0059b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:01.970342 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c98f7-6faa-41de-8428-7906c0059b1d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:01.980957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1298/2624] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:01.985093 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3cf2-8a9f-4647-a890-b583e4130407 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:01.995137 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3cf2-8a9f-4647-a890-b583e4130407 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:01.999709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcae7c-59ff-49f5-91a8-e223f6f202e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:02.000350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcae7c-59ff-49f5-91a8-e223f6f202e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:02.000625 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcae7c-59ff-49f5-91a8-e223f6f202e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:02.000896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcae7c-59ff-49f5-91a8-e223f6f202e1 admin admin] PATH_INFO: `/v3/projects/afbd7e0f51ef41bfac22cf77523a4abe/users/d5108997bb194251bd30aab0d53266ef/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:02.015713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcae7c-59ff-49f5-91a8-e223f6f202e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbd7e0f51ef41bfac22cf77523a4abe, user_id=d5108997bb194251bd30aab0d53266ef, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:02.018399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcae7c-59ff-49f5-91a8-e223f6f202e1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:02.032453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1327/2625] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:01 2022] PUT /identity/v3/projects/afbd7e0f51ef41bfac22cf77523a4abe/users/d5108997bb194251bd30aab0d53266ef/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:02.036393 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c98f7-6faa-41de-8428-7906c0059b1d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:02.046800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c98f7-6faa-41de-8428-7906c0059b1d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:02.103774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0368a-f0ee-4b6c-986a-bee9f1ae225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:02.104386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0368a-f0ee-4b6c-986a-bee9f1ae225d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:02.104688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0368a-f0ee-4b6c-986a-bee9f1ae225d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:02.104962 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0368a-f0ee-4b6c-986a-bee9f1ae225d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:02.105713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0368a-f0ee-4b6c-986a-bee9f1ae225d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:02.107498 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0368a-f0ee-4b6c-986a-bee9f1ae225d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:02.117545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1299/2626] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:02.122109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcae7c-59ff-49f5-91a8-e223f6f202e1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:02.132134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcae7c-59ff-49f5-91a8-e223f6f202e1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:02.137490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:02.138125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:02.138397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:02.138663 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] PATH_INFO: `/v3/projects/afbd7e0f51ef41bfac22cf77523a4abe/users/d5108997bb194251bd30aab0d53266ef/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:02.140536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbd7e0f51ef41bfac22cf77523a4abe, user_id=d5108997bb194251bd30aab0d53266ef, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:02.143175 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:02.152965 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1328/2627] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:02 2022] PUT /identity/v3/projects/afbd7e0f51ef41bfac22cf77523a4abe/users/d5108997bb194251bd30aab0d53266ef/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:45:02.165128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783f6f-6a9c-4885-9116-ed9898a90a32 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:02.165539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783f6f-6a9c-4885-9116-ed9898a90a32 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:02.165848 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783f6f-6a9c-4885-9116-ed9898a90a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:02.172930 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:02.195529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-19783f6f-6a9c-4885-9116-ed9898a90a32 None None] MFA Rules not processed for user `d5108997bb194251bd30aab0d53266ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:02.196734 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-19783f6f-6a9c-4885-9116-ed9898a90a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:02.268539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1300/2628] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:02.279935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:02.282292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:02.339512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff64fae-965e-42c8-9108-ee63d2194624 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:02.343205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52bf59-9b65-4bf3-85c8-4e34d10e1550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:02.343807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52bf59-9b65-4bf3-85c8-4e34d10e1550 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:02.344086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52bf59-9b65-4bf3-85c8-4e34d10e1550 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:02.344358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52bf59-9b65-4bf3-85c8-4e34d10e1550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:02.354079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52bf59-9b65-4bf3-85c8-4e34d10e1550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:02.355565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52bf59-9b65-4bf3-85c8-4e34d10e1550 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:02.366310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1329/2629] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:02.441442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783f6f-6a9c-4885-9116-ed9898a90a32 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:02.452006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783f6f-6a9c-4885-9116-ed9898a90a32 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:02.456140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297d94b-cd43-4b4c-8ed2-71eecde8fb2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:02.456747 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d94b-cd43-4b4c-8ed2-71eecde8fb2e service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:02.457050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d94b-cd43-4b4c-8ed2-71eecde8fb2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:02.457378 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d94b-cd43-4b4c-8ed2-71eecde8fb2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:02.518563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297d94b-cd43-4b4c-8ed2-71eecde8fb2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:02.519983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297d94b-cd43-4b4c-8ed2-71eecde8fb2e service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:02.531303 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1301/2630] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:03.127702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52bf59-9b65-4bf3-85c8-4e34d10e1550 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:03.136531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52bf59-9b65-4bf3-85c8-4e34d10e1550 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:03.140151 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:03.140729 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:03.140999 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:03.141294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:03.143059 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:03.206926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:03.208355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:03.218327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1330/2631] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:03.747089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297d94b-cd43-4b4c-8ed2-71eecde8fb2e service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:03.756021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297d94b-cd43-4b4c-8ed2-71eecde8fb2e service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:03.759928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006c10a-87ba-4d08-a65c-a2b65da167dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:03.760639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006c10a-87ba-4d08-a65c-a2b65da167dd service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:03.760914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006c10a-87ba-4d08-a65c-a2b65da167dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:03.761186 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006c10a-87ba-4d08-a65c-a2b65da167dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:03.827835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006c10a-87ba-4d08-a65c-a2b65da167dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:03.830226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006c10a-87ba-4d08-a65c-a2b65da167dd service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:03.841019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1302/2632] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:05.389589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:05.391106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:05.459720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:05.467929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8e6e-ae4a-4375-8971-d51c3d5b684b service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:05.473326 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:05.474100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:05.474524 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:05.474846 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] PATH_INFO: `/v3/users/9b6ef1675a574d92be3902ee97289421` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:05.475974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6ef1675a574d92be3902ee97289421)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:05.477937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:05.524423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6ef1675a574d92be3902ee97289421'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:05.532086 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6ef1675a574d92be3902ee97289421'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:05.532851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6ef1675a574d92be3902ee97289421'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:05.544951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6ef1675a574d92be3902ee97289421'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:05.545623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6ef1675a574d92be3902ee97289421'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:05.557598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1331/2633] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:05 2022] DELETE /identity/v3/users/9b6ef1675a574d92be3902ee97289421 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:05.562968 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace6b04-9dc3-431d-aec2-881c59d8295e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:05.563465 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace6b04-9dc3-431d-aec2-881c59d8295e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:05.563621 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace6b04-9dc3-431d-aec2-881c59d8295e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:05.583033 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0ace6b04-9dc3-431d-aec2-881c59d8295e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:05.584501 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0ace6b04-9dc3-431d-aec2-881c59d8295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:05.597804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:05.653432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1303/2634] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:05.664711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:05.666156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:05.723967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de0ed2-790f-4e58-8eb6-0604378219a3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:05.727614 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:05.728209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:05.728486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:05.728757 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:05.738235 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:05.739669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:05.749291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1332/2635] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:45:06.022048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace6b04-9dc3-431d-aec2-881c59d8295e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:06.023473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0ace6b04-9dc3-431d-aec2-881c59d8295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:06.038587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace6b04-9dc3-431d-aec2-881c59d8295e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:06.044924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:06.045601 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:06.045910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:06.046264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] PATH_INFO: `/v3/projects/2235b02be77e4eb29499331aa2535421` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:06.047353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2235b02be77e4eb29499331aa2535421)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:06.049571 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:06.095904 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2235b02be77e4eb29499331aa2535421', 'request_id': 'req-e85efb85-0ef8-443b-86b2-fa531c85513e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:06.101510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2235b02be77e4eb29499331aa2535421', 'request_id': 'req-e85efb85-0ef8-443b-86b2-fa531c85513e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:06.107659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2235b02be77e4eb29499331aa2535421', 'request_id': 'req-e85efb85-0ef8-443b-86b2-fa531c85513e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:06.121721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] The token cache is being invalidate because project 2235b02be77e4eb29499331aa2535421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:06.122004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:06.122642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e85efb85-0ef8-443b-86b2-fa531c85513e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:06.124109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1304/2636] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:06 2022] DELETE /identity/v3/projects/2235b02be77e4eb29499331aa2535421 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:07.907909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:07.909347 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:07.922328 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2e774-4426-4d55-8651-b1d48f96cc49 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:07.976938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:07.977578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:07.977856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:07.978125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] PATH_INFO: `/v3/users/2c58a4950da34c189b7a1b82ebc9289d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:07.979242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c58a4950da34c189b7a1b82ebc9289d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:07.981704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:08.002423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:08.027712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c58a4950da34c189b7a1b82ebc9289d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.035538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c58a4950da34c189b7a1b82ebc9289d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.036205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c58a4950da34c189b7a1b82ebc9289d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.048863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c58a4950da34c189b7a1b82ebc9289d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.049525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c58a4950da34c189b7a1b82ebc9289d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.061792 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1333/2637] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:07 2022] DELETE /identity/v3/users/2c58a4950da34c189b7a1b82ebc9289d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:08.066333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acce19-491e-4a6f-a2c0-ab4336a61c27 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:08.066802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acce19-491e-4a6f-a2c0-ab4336a61c27 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:08.066921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acce19-491e-4a6f-a2c0-ab4336a61c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:08.086219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c0acce19-491e-4a6f-a2c0-ab4336a61c27 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:08.087488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c0acce19-491e-4a6f-a2c0-ab4336a61c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:08.149214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1305/2638] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:08.159941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:08.161538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:08.221247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce50e51-f253-4e60-bda4-ae5b29b076cb admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:08.226010 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c2e44-7f94-43ff-bb92-f0c14839c2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:08.226569 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c2e44-7f94-43ff-bb92-f0c14839c2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:08.226845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c2e44-7f94-43ff-bb92-f0c14839c2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:08.227114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c2e44-7f94-43ff-bb92-f0c14839c2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:08.238793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c2e44-7f94-43ff-bb92-f0c14839c2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:08.240308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c2e44-7f94-43ff-bb92-f0c14839c2c9 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:08.252294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1334/2639] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:08.524218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acce19-491e-4a6f-a2c0-ab4336a61c27 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:08.525663 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c0acce19-491e-4a6f-a2c0-ab4336a61c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:08.538719 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acce19-491e-4a6f-a2c0-ab4336a61c27 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:08.543639 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:08.544275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:08.544549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:08.544814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] PATH_INFO: `/v3/projects/2c8155a10bc44935988111c19ccf70ad` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:08.545960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8155a10bc44935988111c19ccf70ad)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:08.548094 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:08.587067 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:08.596008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8155a10bc44935988111c19ccf70ad', 'request_id': 'req-eec1c291-dba5-4061-9b11-5639995e0768'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.601419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8155a10bc44935988111c19ccf70ad', 'request_id': 'req-eec1c291-dba5-4061-9b11-5639995e0768'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.607548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8155a10bc44935988111c19ccf70ad', 'request_id': 'req-eec1c291-dba5-4061-9b11-5639995e0768'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.620287 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] The token cache is being invalidate because project 2c8155a10bc44935988111c19ccf70ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:08.620594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.621200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.622672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1306/2640] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:08 2022] DELETE /identity/v3/projects/2c8155a10bc44935988111c19ccf70ad => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:08.626890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c2e44-7f94-43ff-bb92-f0c14839c2c9 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:08.637335 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c2e44-7f94-43ff-bb92-f0c14839c2c9 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:08.828057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:08.828703 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:08.828975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:08.829633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] PATH_INFO: `/v3/users/a7f16430c5a342c78e0e04a53aaf34f4` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:08.831036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f16430c5a342c78e0e04a53aaf34f4)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:08.832930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:08.875974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:08.882228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f16430c5a342c78e0e04a53aaf34f4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.891439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f16430c5a342c78e0e04a53aaf34f4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.892068 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f16430c5a342c78e0e04a53aaf34f4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.906875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f16430c5a342c78e0e04a53aaf34f4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.907498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f16430c5a342c78e0e04a53aaf34f4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:08.920940 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1335/2641] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:08 2022] DELETE /identity/v3/users/a7f16430c5a342c78e0e04a53aaf34f4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:09.200290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:09.201847 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:09.215662 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1c291-dba5-4061-9b11-5639995e0768 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:09.274868 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:09.275509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:09.275785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:09.276052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] PATH_INFO: `/v3/projects/0c675f2d1be44bdc8923caac3c54d746` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:09.277803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c675f2d1be44bdc8923caac3c54d746)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:09.279750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:09.330754 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c675f2d1be44bdc8923caac3c54d746', 'request_id': 'req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.337269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c675f2d1be44bdc8923caac3c54d746', 'request_id': 'req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.344267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c675f2d1be44bdc8923caac3c54d746', 'request_id': 'req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.357701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] The token cache is being invalidate because project 0c675f2d1be44bdc8923caac3c54d746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:09.358012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.358619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.360419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1307/2642] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:09 2022] DELETE /identity/v3/projects/0c675f2d1be44bdc8923caac3c54d746 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:09.364499 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:09.365924 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:09.378739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b079a6-ef0c-49f7-b1fc-17b9ecce9eeb admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:09.441711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:09.442408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:09.442627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:09.442899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] PATH_INFO: `/v3/users/1b4854f5c7e747c5b887b2c0ff60b2d3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:09.443986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4854f5c7e747c5b887b2c0ff60b2d3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:09.446019 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:09.498679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4854f5c7e747c5b887b2c0ff60b2d3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.506152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4854f5c7e747c5b887b2c0ff60b2d3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.507163 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4854f5c7e747c5b887b2c0ff60b2d3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.520106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4854f5c7e747c5b887b2c0ff60b2d3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.520787 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b78c822-74df-4062-be16-25b90856f768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4854f5c7e747c5b887b2c0ff60b2d3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.534015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1336/2643] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:09 2022] DELETE /identity/v3/users/1b4854f5c7e747c5b887b2c0ff60b2d3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:09.835013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:09.836419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:09.851779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e430f-dd0d-444b-a26d-5e8fe4c85bd0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:09.903545 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:09.910266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:09.911125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:09.911614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:09.911907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] PATH_INFO: `/v3/projects/205ff48977a041d281cd561fea437957` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:09.913517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=205ff48977a041d281cd561fea437957)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:09.915721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:09.962025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205ff48977a041d281cd561fea437957', 'request_id': 'req-f1ade614-d613-4ff0-be19-1121e43a82ab'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.967160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205ff48977a041d281cd561fea437957', 'request_id': 'req-f1ade614-d613-4ff0-be19-1121e43a82ab'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.973107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205ff48977a041d281cd561fea437957', 'request_id': 'req-f1ade614-d613-4ff0-be19-1121e43a82ab'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.985136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] The token cache is being invalidate because project 205ff48977a041d281cd561fea437957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:09.985457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.986052 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f1ade614-d613-4ff0-be19-1121e43a82ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:09.987505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1308/2644] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:09 2022] DELETE /identity/v3/projects/205ff48977a041d281cd561fea437957 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:09.992656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95a9f-e5ab-4cae-9161-ac7dc1c67556 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:09.993138 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95a9f-e5ab-4cae-9161-ac7dc1c67556 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:09.993260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95a9f-e5ab-4cae-9161-ac7dc1c67556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.012776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a1e95a9f-e5ab-4cae-9161-ac7dc1c67556 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:10.014002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a1e95a9f-e5ab-4cae-9161-ac7dc1c67556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:10.045372 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:10.073120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1337/2645] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:10.082670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d36b-abef-41f6-81f3-a4f2bbc7251d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.082953 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d36b-abef-41f6-81f3-a4f2bbc7251d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.083224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d36b-abef-41f6-81f3-a4f2bbc7251d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.103642 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-56c9d36b-abef-41f6-81f3-a4f2bbc7251d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:10.105441 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-56c9d36b-abef-41f6-81f3-a4f2bbc7251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:10.113851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1309/2646] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:10.118039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e95a9f-e5ab-4cae-9161-ac7dc1c67556 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:10.128061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e95a9f-e5ab-4cae-9161-ac7dc1c67556 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:10.132744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:10.133379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.133655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.133921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.134601 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:45:10.134891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:10.136531 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:10.147930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1338/2647] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:45:10.152124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9d36b-abef-41f6-81f3-a4f2bbc7251d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:10.161170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9d36b-abef-41f6-81f3-a4f2bbc7251d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:10.165718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a86bbb-7275-4d7a-a8c8-ad1eaec648da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:10.166316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a86bbb-7275-4d7a-a8c8-ad1eaec648da admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.166588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a86bbb-7275-4d7a-a8c8-ad1eaec648da admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.166851 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a86bbb-7275-4d7a-a8c8-ad1eaec648da admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.167973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a86bbb-7275-4d7a-a8c8-ad1eaec648da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:10.169892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a86bbb-7275-4d7a-a8c8-ad1eaec648da admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:10.198839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1310/2648] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:10 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:10.202391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:10.212194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdba77-e3be-4512-b942-6cac1c427c22 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:10.267002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50154504-63b6-43db-8de4-9825bf93d955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:10.267604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50154504-63b6-43db-8de4-9825bf93d955 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.267878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50154504-63b6-43db-8de4-9825bf93d955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.268148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50154504-63b6-43db-8de4-9825bf93d955 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.268764 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50154504-63b6-43db-8de4-9825bf93d955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:10.270699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50154504-63b6-43db-8de4-9825bf93d955 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:10.292851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1339/2649] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:10 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:10.297319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a86bbb-7275-4d7a-a8c8-ad1eaec648da admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:10.307519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a86bbb-7275-4d7a-a8c8-ad1eaec648da admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:10.312585 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21611b3-d88f-4352-80a6-e64b7b243d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:10.313177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21611b3-d88f-4352-80a6-e64b7b243d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.313480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21611b3-d88f-4352-80a6-e64b7b243d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.313748 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21611b3-d88f-4352-80a6-e64b7b243d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.314387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21611b3-d88f-4352-80a6-e64b7b243d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:10.315960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21611b3-d88f-4352-80a6-e64b7b243d86 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:10.326154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1311/2650] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:10.330178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50154504-63b6-43db-8de4-9825bf93d955 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:10.340155 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50154504-63b6-43db-8de4-9825bf93d955 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:10.344686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e61788-bfd5-4752-ba6c-92cc5bade9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:10.345295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61788-bfd5-4752-ba6c-92cc5bade9db admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.345571 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61788-bfd5-4752-ba6c-92cc5bade9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.345836 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61788-bfd5-4752-ba6c-92cc5bade9db admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.346467 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e61788-bfd5-4752-ba6c-92cc5bade9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:10.348032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e61788-bfd5-4752-ba6c-92cc5bade9db admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:10.358369 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1340/2651] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:10.363199 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21611b3-d88f-4352-80a6-e64b7b243d86 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:10.374028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21611b3-d88f-4352-80a6-e64b7b243d86 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:10.378690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:10.379336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.379611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.379876 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] PATH_INFO: `/v3/projects/64d71e75536146e8aa2ccba12c266337/users/8a865dc406ef4650acaeb75f5b0f0f13/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.394714 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d71e75536146e8aa2ccba12c266337, user_id=8a865dc406ef4650acaeb75f5b0f0f13, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:10.397385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:10.409924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1312/2652] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:10 2022] PUT /identity/v3/projects/64d71e75536146e8aa2ccba12c266337/users/8a865dc406ef4650acaeb75f5b0f0f13/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:10.420249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7b2db-bfd2-44dd-9e76-badee50db47d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.420530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7b2db-bfd2-44dd-9e76-badee50db47d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.420797 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7b2db-bfd2-44dd-9e76-badee50db47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.448544 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2ff7b2db-bfd2-44dd-9e76-badee50db47d None None] MFA Rules not processed for user `8a865dc406ef4650acaeb75f5b0f0f13`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:10.449845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2ff7b2db-bfd2-44dd-9e76-badee50db47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:10.514048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1341/2653] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:10.524685 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:10.526432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:10.584496 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3526fda-39a5-4b56-90ee-6b190aae6e49 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:10.588129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed5963-74ec-4b47-8042-7aefe705990c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:10.588736 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed5963-74ec-4b47-8042-7aefe705990c service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:10.589012 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed5963-74ec-4b47-8042-7aefe705990c service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:10.589316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed5963-74ec-4b47-8042-7aefe705990c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:10.597450 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed5963-74ec-4b47-8042-7aefe705990c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:10.598856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed5963-74ec-4b47-8042-7aefe705990c service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:10.607488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1313/2654] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:19.981445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7b2db-bfd2-44dd-9e76-badee50db47d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:19.982927 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2ff7b2db-bfd2-44dd-9e76-badee50db47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:20.021695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:20.052497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7b2db-bfd2-44dd-9e76-badee50db47d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:20.057336 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:20.058445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:20.058720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:20.058985 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] PATH_INFO: `/v3/users/fc65bfeeff464616886cddb8713a5354` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:20.060087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc65bfeeff464616886cddb8713a5354)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:20.062041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:20.107590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc65bfeeff464616886cddb8713a5354'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.115539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc65bfeeff464616886cddb8713a5354'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.116345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc65bfeeff464616886cddb8713a5354'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.129122 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc65bfeeff464616886cddb8713a5354'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.129760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc65bfeeff464616886cddb8713a5354'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.141910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1342/2655] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:19 2022] DELETE /identity/v3/users/fc65bfeeff464616886cddb8713a5354 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:20.146476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf62a8-be32-47af-9bf1-d62ae36023ec None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:20.146944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf62a8-be32-47af-9bf1-d62ae36023ec None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:20.147091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf62a8-be32-47af-9bf1-d62ae36023ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:20.165827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f1cf62a8-be32-47af-9bf1-d62ae36023ec None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:20.167034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f1cf62a8-be32-47af-9bf1-d62ae36023ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:20.187705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:20.227041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1314/2656] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:20.236678 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:20.238368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:20.298724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a11393-4dc9-415a-afe7-8f80b22c45f2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:20.303050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf60e13-1d92-4dd4-90f3-70c69d09c16d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:20.303684 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf60e13-1d92-4dd4-90f3-70c69d09c16d service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:20.303987 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf60e13-1d92-4dd4-90f3-70c69d09c16d service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:20.304254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf60e13-1d92-4dd4-90f3-70c69d09c16d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:20.314251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf60e13-1d92-4dd4-90f3-70c69d09c16d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:20.315680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf60e13-1d92-4dd4-90f3-70c69d09c16d service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:20.325705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1343/2657] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:20.661401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf62a8-be32-47af-9bf1-d62ae36023ec None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:20.664205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f1cf62a8-be32-47af-9bf1-d62ae36023ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:20.682136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf62a8-be32-47af-9bf1-d62ae36023ec None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:20.688783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:20.689461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:20.689741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:20.690087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] PATH_INFO: `/v3/projects/29af2ad658b040869214a908ed82b517` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:20.702229 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=29af2ad658b040869214a908ed82b517)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:20.704319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:20.754803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29af2ad658b040869214a908ed82b517', 'request_id': 'req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.760302 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29af2ad658b040869214a908ed82b517', 'request_id': 'req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.767104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29af2ad658b040869214a908ed82b517', 'request_id': 'req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.779419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] The token cache is being invalidate because project 29af2ad658b040869214a908ed82b517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:20.779704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.780320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.781880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1315/2658] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:20 2022] DELETE /identity/v3/projects/29af2ad658b040869214a908ed82b517 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:20.785835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf60e13-1d92-4dd4-90f3-70c69d09c16d service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:20.795360 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf60e13-1d92-4dd4-90f3-70c69d09c16d service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:20.848822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:20.861842 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:20.862582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:20.863321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:20.863866 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] PATH_INFO: `/v3/users/d5108997bb194251bd30aab0d53266ef` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:20.865423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5108997bb194251bd30aab0d53266ef)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:20.867361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:20.919767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5108997bb194251bd30aab0d53266ef'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.927492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5108997bb194251bd30aab0d53266ef'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.928196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5108997bb194251bd30aab0d53266ef'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.941491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5108997bb194251bd30aab0d53266ef'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.942192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-58d45acd-8759-47c1-931e-dfbe1ff02e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5108997bb194251bd30aab0d53266ef'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:20.955951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1344/2659] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:20 2022] DELETE /identity/v3/users/d5108997bb194251bd30aab0d53266ef => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:21.230767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:21.232225 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:21.249192 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45054c-6d20-4bb8-a63c-912f7e5f7dbd admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:21.305753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:21.306314 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.306589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.306858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] PATH_INFO: `/v3/projects/afbd7e0f51ef41bfac22cf77523a4abe` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.308591 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] RBAC: Authorizing `identity:delete_project(project_id=afbd7e0f51ef41bfac22cf77523a4abe)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:21.310519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:21.356028 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afbd7e0f51ef41bfac22cf77523a4abe', 'request_id': 'req-cffa3bb5-1083-42d2-8b56-6853cce8623d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:21.358004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:21.362304 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afbd7e0f51ef41bfac22cf77523a4abe', 'request_id': 'req-cffa3bb5-1083-42d2-8b56-6853cce8623d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:21.369565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afbd7e0f51ef41bfac22cf77523a4abe', 'request_id': 'req-cffa3bb5-1083-42d2-8b56-6853cce8623d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:21.381987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] The token cache is being invalidate because project afbd7e0f51ef41bfac22cf77523a4abe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:21.382269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:21.382870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cffa3bb5-1083-42d2-8b56-6853cce8623d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:21.384321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1316/2660] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:21 2022] DELETE /identity/v3/projects/afbd7e0f51ef41bfac22cf77523a4abe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:21.389152 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc202e-dadb-441b-9559-68ce74ad2cc9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.389669 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc202e-dadb-441b-9559-68ce74ad2cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.389770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc202e-dadb-441b-9559-68ce74ad2cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.409898 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-68fc202e-dadb-441b-9559-68ce74ad2cc9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:21.411106 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-68fc202e-dadb-441b-9559-68ce74ad2cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:21.468679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1345/2661] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:21.476387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d54c9-3575-4878-896f-a1c6b8d98776 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.476672 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d54c9-3575-4878-896f-a1c6b8d98776 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.476948 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d54c9-3575-4878-896f-a1c6b8d98776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.497826 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5c4d54c9-3575-4878-896f-a1c6b8d98776 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:21.499027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5c4d54c9-3575-4878-896f-a1c6b8d98776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:21.507116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1317/2662] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:21.511292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc202e-dadb-441b-9559-68ce74ad2cc9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:21.520795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc202e-dadb-441b-9559-68ce74ad2cc9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:21.525261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:21.525850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.526127 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.526896 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.527530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:45:21.527823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:21.529432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:21.540530 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1346/2663] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:45:21.545330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d54c9-3575-4878-896f-a1c6b8d98776 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:21.554296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d54c9-3575-4878-896f-a1c6b8d98776 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:21.559190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b36e9-6332-4b25-960e-eeedd134e323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:21.559730 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b36e9-6332-4b25-960e-eeedd134e323 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.560006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b36e9-6332-4b25-960e-eeedd134e323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.560279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b36e9-6332-4b25-960e-eeedd134e323 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.560929 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b36e9-6332-4b25-960e-eeedd134e323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:21.562912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b36e9-6332-4b25-960e-eeedd134e323 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:21.595323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1318/2664] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:21 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:21.599516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:21.610049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43a20f-5b43-46a4-836f-159dff80b67b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:21.664977 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1822c-7142-4d8b-b3a5-a8952790e9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:21.665610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1822c-7142-4d8b-b3a5-a8952790e9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.665902 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1822c-7142-4d8b-b3a5-a8952790e9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.666156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1822c-7142-4d8b-b3a5-a8952790e9e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.666776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1822c-7142-4d8b-b3a5-a8952790e9e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:21.669260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1822c-7142-4d8b-b3a5-a8952790e9e0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:21.677783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:21.690456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1347/2665] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:21 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:21.694228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b36e9-6332-4b25-960e-eeedd134e323 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:21.704769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b36e9-6332-4b25-960e-eeedd134e323 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:21.709373 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea4e9d-1884-46ff-be8b-baa283cfcfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:21.710039 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4e9d-1884-46ff-be8b-baa283cfcfac admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.710319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4e9d-1884-46ff-be8b-baa283cfcfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.710588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea4e9d-1884-46ff-be8b-baa283cfcfac admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.711222 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea4e9d-1884-46ff-be8b-baa283cfcfac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:21.712966 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea4e9d-1884-46ff-be8b-baa283cfcfac admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:21.728509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1319/2666] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:45:21.733363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1822c-7142-4d8b-b3a5-a8952790e9e0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:21.744560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1822c-7142-4d8b-b3a5-a8952790e9e0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:21.749882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe572e-c9f1-4600-b894-e7ec2a6cd754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:21.750550 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe572e-c9f1-4600-b894-e7ec2a6cd754 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.750848 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe572e-c9f1-4600-b894-e7ec2a6cd754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.751116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe572e-c9f1-4600-b894-e7ec2a6cd754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.751772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe572e-c9f1-4600-b894-e7ec2a6cd754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:21.753686 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe572e-c9f1-4600-b894-e7ec2a6cd754 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:21.766772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1348/2667] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:45:21.771137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea4e9d-1884-46ff-be8b-baa283cfcfac admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:21.797872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea4e9d-1884-46ff-be8b-baa283cfcfac admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:21.804839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:21.805673 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.805807 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.806085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] PATH_INFO: `/v3/projects/7babfa044a95430f8745620b5b6016fc/users/bffb715b24934f72944c7d108293001a/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.825914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7babfa044a95430f8745620b5b6016fc, user_id=bffb715b24934f72944c7d108293001a, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:21.828602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:21.842694 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1320/2668] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:21 2022] PUT /identity/v3/projects/7babfa044a95430f8745620b5b6016fc/users/bffb715b24934f72944c7d108293001a/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:21.853357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4994d-9b86-4f92-b336-e751c8bc774c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:21.853660 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4994d-9b86-4f92-b336-e751c8bc774c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:21.853951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4994d-9b86-4f92-b336-e751c8bc774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:21.881645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-66d4994d-9b86-4f92-b336-e751c8bc774c None None] MFA Rules not processed for user `bffb715b24934f72944c7d108293001a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:21.882874 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-66d4994d-9b86-4f92-b336-e751c8bc774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:21.956878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1349/2669] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:21.967576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:21.969633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:22.029780 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b12ec-712f-4ad3-a1db-9dd7840a9c01 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:22.033390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1e7ae-43f9-45d3-9321-b5bb7cf47d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:22.034019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1e7ae-43f9-45d3-9321-b5bb7cf47d47 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:22.034290 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1e7ae-43f9-45d3-9321-b5bb7cf47d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:22.034554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1e7ae-43f9-45d3-9321-b5bb7cf47d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:22.042806 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1e7ae-43f9-45d3-9321-b5bb7cf47d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:22.044210 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1e7ae-43f9-45d3-9321-b5bb7cf47d47 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:22.052799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1321/2670] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:22.862542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4994d-9b86-4f92-b336-e751c8bc774c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:22.864317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-66d4994d-9b86-4f92-b336-e751c8bc774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:22.931501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4994d-9b86-4f92-b336-e751c8bc774c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:22.936615 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:22.937255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:22.937539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:22.937807 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] PATH_INFO: `/v3/users/8a865dc406ef4650acaeb75f5b0f0f13` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:22.938911 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a865dc406ef4650acaeb75f5b0f0f13)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:22.940823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:22.984761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a865dc406ef4650acaeb75f5b0f0f13'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:22.992607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a865dc406ef4650acaeb75f5b0f0f13'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:22.993553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a865dc406ef4650acaeb75f5b0f0f13'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:23.007162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a865dc406ef4650acaeb75f5b0f0f13'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:23.008459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a865dc406ef4650acaeb75f5b0f0f13'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:23.009945 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:23.027453 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1350/2671] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:22 2022] DELETE /identity/v3/users/8a865dc406ef4650acaeb75f5b0f0f13 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:23.031858 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f6ae-e1db-40ec-9d9a-ec21ea8137c3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.032328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f6ae-e1db-40ec-9d9a-ec21ea8137c3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.032449 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f6ae-e1db-40ec-9d9a-ec21ea8137c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.051046 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-55a6f6ae-e1db-40ec-9d9a-ec21ea8137c3 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:23.052238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-55a6f6ae-e1db-40ec-9d9a-ec21ea8137c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:23.109170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1322/2672] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:23.119188 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:23.121111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:23.177479 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7c63-8f24-4232-a771-91bb2ada32f2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:23.181389 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf16b77-a6e2-47eb-b994-adb7d86fdc11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:23.181989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf16b77-a6e2-47eb-b994-adb7d86fdc11 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.182261 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf16b77-a6e2-47eb-b994-adb7d86fdc11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.183023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf16b77-a6e2-47eb-b994-adb7d86fdc11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.192123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf16b77-a6e2-47eb-b994-adb7d86fdc11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:23.193573 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf16b77-a6e2-47eb-b994-adb7d86fdc11 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:23.203412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1351/2673] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:45:23.468958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6f6ae-e1db-40ec-9d9a-ec21ea8137c3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:23.470519 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-55a6f6ae-e1db-40ec-9d9a-ec21ea8137c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:23.484645 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6f6ae-e1db-40ec-9d9a-ec21ea8137c3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:23.490076 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:23.490906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.491178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.491445 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] PATH_INFO: `/v3/projects/64d71e75536146e8aa2ccba12c266337` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.492599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d71e75536146e8aa2ccba12c266337)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:23.494654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:23.511376 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:23.541486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d71e75536146e8aa2ccba12c266337', 'request_id': 'req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:23.547284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d71e75536146e8aa2ccba12c266337', 'request_id': 'req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:23.553292 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d71e75536146e8aa2ccba12c266337', 'request_id': 'req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:23.565243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] The token cache is being invalidate because project 64d71e75536146e8aa2ccba12c266337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:23.565544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:23.566165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6cd53d73-cf9e-4dac-9058-2d8723f3f29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:23.567600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1323/2674] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:23 2022] DELETE /identity/v3/projects/64d71e75536146e8aa2ccba12c266337 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:23.572447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2752b-0b10-4737-8e26-500936bf7c00 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.572741 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2752b-0b10-4737-8e26-500936bf7c00 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.572989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2752b-0b10-4737-8e26-500936bf7c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.592626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3dc2752b-0b10-4737-8e26-500936bf7c00 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:23.593844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3dc2752b-0b10-4737-8e26-500936bf7c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:23.651946 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1352/2675] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:23.660651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d25f-f55a-4c9d-8310-4a8248152819 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.661006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d25f-f55a-4c9d-8310-4a8248152819 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.661392 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d25f-f55a-4c9d-8310-4a8248152819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.680751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f8d5d25f-f55a-4c9d-8310-4a8248152819 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:23.681978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f8d5d25f-f55a-4c9d-8310-4a8248152819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:23.689779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1324/2676] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:23.694017 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2752b-0b10-4737-8e26-500936bf7c00 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:23.704029 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2752b-0b10-4737-8e26-500936bf7c00 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:23.708562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:23.709148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.709451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.709720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.710351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:45:23.710643 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:23.712217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:23.723989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1353/2677] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:45:23.728103 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5d25f-f55a-4c9d-8310-4a8248152819 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:23.737275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5d25f-f55a-4c9d-8310-4a8248152819 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:23.742190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cda53-4b0b-4ab4-b397-7b25f8fff812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:23.742781 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cda53-4b0b-4ab4-b397-7b25f8fff812 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.743058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cda53-4b0b-4ab4-b397-7b25f8fff812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.743330 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cda53-4b0b-4ab4-b397-7b25f8fff812 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.743957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cda53-4b0b-4ab4-b397-7b25f8fff812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:23.745865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cda53-4b0b-4ab4-b397-7b25f8fff812 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:23.772473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1325/2678] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:23 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:23.777172 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:23.787070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c85c2a-516b-4eae-9723-55772e65acbc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:23.841368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa9fc0-5701-4810-a8b4-9ee6f7d32bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:23.841991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa9fc0-5701-4810-a8b4-9ee6f7d32bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.842265 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa9fc0-5701-4810-a8b4-9ee6f7d32bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.842551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa9fc0-5701-4810-a8b4-9ee6f7d32bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.843195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa9fc0-5701-4810-a8b4-9ee6f7d32bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:23.845084 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa9fc0-5701-4810-a8b4-9ee6f7d32bd1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:23.865835 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1354/2679] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:23 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:23.869884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cda53-4b0b-4ab4-b397-7b25f8fff812 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:23.880474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cda53-4b0b-4ab4-b397-7b25f8fff812 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:23.884919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b28e4-6907-40d4-9be9-1d66941e808a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:23.885536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b28e4-6907-40d4-9be9-1d66941e808a admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.886447 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b28e4-6907-40d4-9be9-1d66941e808a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.886717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b28e4-6907-40d4-9be9-1d66941e808a admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.887367 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b28e4-6907-40d4-9be9-1d66941e808a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:23.888969 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b28e4-6907-40d4-9be9-1d66941e808a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:23.898564 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1326/2680] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:23.902715 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa9fc0-5701-4810-a8b4-9ee6f7d32bd1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:23.912474 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa9fc0-5701-4810-a8b4-9ee6f7d32bd1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:23.917101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20074ba8-2938-4665-a2f0-b30aa827972a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:23.917713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20074ba8-2938-4665-a2f0-b30aa827972a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.917990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20074ba8-2938-4665-a2f0-b30aa827972a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.918259 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20074ba8-2938-4665-a2f0-b30aa827972a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.918891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20074ba8-2938-4665-a2f0-b30aa827972a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:23.920466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20074ba8-2938-4665-a2f0-b30aa827972a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:23.922473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:23.933405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1355/2681] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:23.936561 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b28e4-6907-40d4-9be9-1d66941e808a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:23.946351 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b28e4-6907-40d4-9be9-1d66941e808a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:23.951455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:23.952082 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.952354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.952619 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] PATH_INFO: `/v3/projects/bf531c40d3f74fa6aaf3d26794dca09c/users/0968c56b8be242918903711961b6d5c0/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:23.966837 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf531c40d3f74fa6aaf3d26794dca09c, user_id=0968c56b8be242918903711961b6d5c0, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:23.969480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:23.981741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1327/2682] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:23 2022] PUT /identity/v3/projects/bf531c40d3f74fa6aaf3d26794dca09c/users/0968c56b8be242918903711961b6d5c0/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:45:23.986725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e90d68-ea73-4352-8a7a-4ce26ae84dde None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:23.987021 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e90d68-ea73-4352-8a7a-4ce26ae84dde None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:23.987277 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e90d68-ea73-4352-8a7a-4ce26ae84dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:24.006990 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f7e90d68-ea73-4352-8a7a-4ce26ae84dde None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:24.008194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f7e90d68-ea73-4352-8a7a-4ce26ae84dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:24.068331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1356/2683] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:24.078529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:24.080225 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:24.137275 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ade5bb-6424-40ad-87ab-92cf4937c0d7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:24.140841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:24.141468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:24.141742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:24.142010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:24.150993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:24.152474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:24.162147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1328/2684] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:24.783301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356097e2-90c0-4853-8f51-3a33d0991ada None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:24.783579 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356097e2-90c0-4853-8f51-3a33d0991ada None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:24.783845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356097e2-90c0-4853-8f51-3a33d0991ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:24.811275 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-356097e2-90c0-4853-8f51-3a33d0991ada None None] MFA Rules not processed for user `0968c56b8be242918903711961b6d5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:24.812464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-356097e2-90c0-4853-8f51-3a33d0991ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:24.881077 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1357/2685] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:24.892471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:24.893880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:24.941196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:24.956365 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79942fd8-86d9-4873-8c5b-5ed1222debe0 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:24.959956 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:24.960565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:24.960841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:24.961112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:24.970676 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:24.972135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:24.980949 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1329/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:26.365782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d28bd7-f1e7-478a-99e0-3ef43b9d93a3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:26.366357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d28bd7-f1e7-478a-99e0-3ef43b9d93a3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:26.366357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d28bd7-f1e7-478a-99e0-3ef43b9d93a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:26.384481 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-42d28bd7-f1e7-478a-99e0-3ef43b9d93a3 None None] MFA Rules not processed for user `0968c56b8be242918903711961b6d5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:26.385779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-42d28bd7-f1e7-478a-99e0-3ef43b9d93a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:26.392460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1358/2687] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:27.169490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:27.170931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:27.185644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3b75f-b069-46a3-ad05-e84419121f07 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:27.190200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:27.190835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:27.191113 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:27.191386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] PATH_INFO: `/v3/users/bffb715b24934f72944c7d108293001a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:27.192508 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bffb715b24934f72944c7d108293001a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:27.194475 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:27.246998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bffb715b24934f72944c7d108293001a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.256220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bffb715b24934f72944c7d108293001a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.257014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bffb715b24934f72944c7d108293001a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.265010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bffb715b24934f72944c7d108293001a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.279759 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bffb715b24934f72944c7d108293001a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.287066 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1330/2688] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:27 2022] DELETE /identity/v3/users/bffb715b24934f72944c7d108293001a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:27.291696 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fed3b0-518f-4d57-ab0f-0c4497eee021 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:27.292510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fed3b0-518f-4d57-ab0f-0c4497eee021 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:27.292779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fed3b0-518f-4d57-ab0f-0c4497eee021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:27.313253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e6fed3b0-518f-4d57-ab0f-0c4497eee021 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:27.314734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e6fed3b0-518f-4d57-ab0f-0c4497eee021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:27.378853 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1359/2689] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:27.388670 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:27.390491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:27.452990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e266-1093-4784-b0d8-8a0822ba05f6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:27.456831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d258c41-827c-4010-8313-76819438b89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:27.457470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d258c41-827c-4010-8313-76819438b89c service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:27.457745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d258c41-827c-4010-8313-76819438b89c service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:27.458010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d258c41-827c-4010-8313-76819438b89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:27.467830 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d258c41-827c-4010-8313-76819438b89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:27.469442 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d258c41-827c-4010-8313-76819438b89c service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:27.479125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1331/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:27.756312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fed3b0-518f-4d57-ab0f-0c4497eee021 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:27.757810 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e6fed3b0-518f-4d57-ab0f-0c4497eee021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:27.770014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fed3b0-518f-4d57-ab0f-0c4497eee021 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:27.774845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:27.775466 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:27.776003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:27.777003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] PATH_INFO: `/v3/projects/7babfa044a95430f8745620b5b6016fc` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:27.778413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7babfa044a95430f8745620b5b6016fc)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:27.780546 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:27.805966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:27.850566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7babfa044a95430f8745620b5b6016fc', 'request_id': 'req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.857858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7babfa044a95430f8745620b5b6016fc', 'request_id': 'req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.864827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7babfa044a95430f8745620b5b6016fc', 'request_id': 'req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.884763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] The token cache is being invalidate because project 7babfa044a95430f8745620b5b6016fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:27.886118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.886957 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f7d8e9fe-0acf-49dc-9db6-ecdfc6a99d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:27.889222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1360/2691] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:27 2022] DELETE /identity/v3/projects/7babfa044a95430f8745620b5b6016fc => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:27.894108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4917179-00ad-41fe-a984-879cc11d43ab None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:27.894406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4917179-00ad-41fe-a984-879cc11d43ab None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:27.894679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4917179-00ad-41fe-a984-879cc11d43ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:27.929384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d4917179-00ad-41fe-a984-879cc11d43ab None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:27.931455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d4917179-00ad-41fe-a984-879cc11d43ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:28.024520 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:28.053732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1332/2692] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:28.057156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c066e9-4277-46a7-b40b-de8e26803bce None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.058405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c066e9-4277-46a7-b40b-de8e26803bce None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.058676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c066e9-4277-46a7-b40b-de8e26803bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.093498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-69c066e9-4277-46a7-b40b-de8e26803bce None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:28.095117 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-69c066e9-4277-46a7-b40b-de8e26803bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:28.103289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1361/2693] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:28.109396 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4917179-00ad-41fe-a984-879cc11d43ab None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.121137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4917179-00ad-41fe-a984-879cc11d43ab None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.126769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.127473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.127822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.128147 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.128911 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:45:28.129340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.131009 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.145746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1333/2694] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:45:28.150289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c066e9-4277-46a7-b40b-de8e26803bce None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.165363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c066e9-4277-46a7-b40b-de8e26803bce None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.170490 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.171181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.171363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.171821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.172497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.174598 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.207416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1362/2695] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:28 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:28.211740 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.228914 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a3c50-e828-4014-9ad6-4343124d81c9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.252236 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.254627 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:28.311271 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889393f-455e-49bb-9fa6-d6fedb3d79db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.311958 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889393f-455e-49bb-9fa6-d6fedb3d79db admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.312238 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889393f-455e-49bb-9fa6-d6fedb3d79db admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.312509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889393f-455e-49bb-9fa6-d6fedb3d79db admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.313145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889393f-455e-49bb-9fa6-d6fedb3d79db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.315183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889393f-455e-49bb-9fa6-d6fedb3d79db admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.331770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2414e-1d1e-4b8b-94b2-afe99ec00d2d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.337666 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.338391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1334/2696] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:28 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:28.338767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.339049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.339318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] PATH_INFO: `/v3/users/903ecebf5c9743118fb5eb2c71b949a5` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.342042 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=903ecebf5c9743118fb5eb2c71b949a5)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.344000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.346514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889393f-455e-49bb-9fa6-d6fedb3d79db admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.360319 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889393f-455e-49bb-9fa6-d6fedb3d79db admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.365201 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa780ba8-bd7f-4767-ab8d-a40b990e78eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.365856 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780ba8-bd7f-4767-ab8d-a40b990e78eb admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.366133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780ba8-bd7f-4767-ab8d-a40b990e78eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.366403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780ba8-bd7f-4767-ab8d-a40b990e78eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.367044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa780ba8-bd7f-4767-ab8d-a40b990e78eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.368643 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa780ba8-bd7f-4767-ab8d-a40b990e78eb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.384728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1335/2697] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:28.386480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa780ba8-bd7f-4767-ab8d-a40b990e78eb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.397632 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa780ba8-bd7f-4767-ab8d-a40b990e78eb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.400545 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '903ecebf5c9743118fb5eb2c71b949a5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:28.403539 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.404507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.404928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.405220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.405866 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.407476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.409221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '903ecebf5c9743118fb5eb2c71b949a5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:28.410002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '903ecebf5c9743118fb5eb2c71b949a5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:28.418701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1336/2698] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:28.423485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.425592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:28.426405 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '903ecebf5c9743118fb5eb2c71b949a5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:28.427254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e40c3da6-fb1b-416a-80ec-a93b55ade5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '903ecebf5c9743118fb5eb2c71b949a5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:28.433823 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:28.442670 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1363/2699] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:28 2022] DELETE /identity/v3/users/903ecebf5c9743118fb5eb2c71b949a5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:45:28.443576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523f19d-644d-4017-aca3-7e1a41111153 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.447555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ff8ab-c7af-4d3c-a5e6-ac4e76160cd9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.448769 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ff8ab-c7af-4d3c-a5e6-ac4e76160cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.449157 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ff8ab-c7af-4d3c-a5e6-ac4e76160cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.469012 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fb2ff8ab-c7af-4d3c-a5e6-ac4e76160cd9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:28.470241 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fb2ff8ab-c7af-4d3c-a5e6-ac4e76160cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:28.517801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029fd16-1a79-48fa-8125-074349710f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.518603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029fd16-1a79-48fa-8125-074349710f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.518957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029fd16-1a79-48fa-8125-074349710f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.519308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029fd16-1a79-48fa-8125-074349710f58 admin admin] PATH_INFO: `/v3/projects/2cc911cd994a49e99bae085ad2f6defa/users/a4e2ef58773a46b998162cb555acdbda/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.532536 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1364/2700] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:28.536022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029fd16-1a79-48fa-8125-074349710f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc911cd994a49e99bae085ad2f6defa, user_id=a4e2ef58773a46b998162cb555acdbda, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.539026 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029fd16-1a79-48fa-8125-074349710f58 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.543420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ff8ab-c7af-4d3c-a5e6-ac4e76160cd9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.545146 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fb2ff8ab-c7af-4d3c-a5e6-ac4e76160cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:28.553978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1337/2701] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:28 2022] PUT /identity/v3/projects/2cc911cd994a49e99bae085ad2f6defa/users/a4e2ef58773a46b998162cb555acdbda/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:28.565770 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42584180-8a2e-4355-a536-8648f389e1e7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.566090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42584180-8a2e-4355-a536-8648f389e1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.566316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42584180-8a2e-4355-a536-8648f389e1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.599913 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-42584180-8a2e-4355-a536-8648f389e1e7 None None] MFA Rules not processed for user `a4e2ef58773a46b998162cb555acdbda`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:28.602123 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-42584180-8a2e-4355-a536-8648f389e1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:28.613725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ff8ab-c7af-4d3c-a5e6-ac4e76160cd9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.634918 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:28.674122 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1338/2702] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:28.675554 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.676487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.676487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.676774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.687300 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42584180-8a2e-4355-a536-8648f389e1e7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:28.689846 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-42584180-8a2e-4355-a536-8648f389e1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:28.744558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.746582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.756805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1365/2703] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:45:28.767333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42584180-8a2e-4355-a536-8648f389e1e7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:28.771463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b152f9-bd16-4df9-8b36-b6e28d268fe9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce851cf0509947e783dbd5b4809bc897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:28.772054 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b152f9-bd16-4df9-8b36-b6e28d268fe9 service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:28.772328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b152f9-bd16-4df9-8b36-b6e28d268fe9 service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:28.772595 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b152f9-bd16-4df9-8b36-b6e28d268fe9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:28.782485 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b152f9-bd16-4df9-8b36-b6e28d268fe9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:28.784053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b152f9-bd16-4df9-8b36-b6e28d268fe9 service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:28.795336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1339/2704] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:45:29.085000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:29.086321 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:29.094489 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:29.102568 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee3461-43f9-4e69-a6e4-e1cbb5fd8b80 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:29.108604 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:29.109526 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:29.109871 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:29.110217 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] PATH_INFO: `/v3/projects/145b73a4868e4b50a6ed18ef69128f3c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:29.113952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=145b73a4868e4b50a6ed18ef69128f3c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:29.116112 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:29.175577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145b73a4868e4b50a6ed18ef69128f3c', 'request_id': 'req-47952608-a1f9-486a-9579-97419c4d2608'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.184382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145b73a4868e4b50a6ed18ef69128f3c', 'request_id': 'req-47952608-a1f9-486a-9579-97419c4d2608'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.191423 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145b73a4868e4b50a6ed18ef69128f3c', 'request_id': 'req-47952608-a1f9-486a-9579-97419c4d2608'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.207409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] The token cache is being invalidate because project 145b73a4868e4b50a6ed18ef69128f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:29.207704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.208991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.210528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1366/2705] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:29 2022] DELETE /identity/v3/projects/145b73a4868e4b50a6ed18ef69128f3c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:29.215135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b152f9-bd16-4df9-8b36-b6e28d268fe9 service glance] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:29.228816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b152f9-bd16-4df9-8b36-b6e28d268fe9 service glance] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:29.302935 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:29.303577 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:29.303849 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:29.304116 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] PATH_INFO: `/v3/users/12596bac740142a99724a4bfd119bc77` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:29.305256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12596bac740142a99724a4bfd119bc77)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:29.307180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:29.368337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12596bac740142a99724a4bfd119bc77'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.381476 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12596bac740142a99724a4bfd119bc77'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.382458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12596bac740142a99724a4bfd119bc77'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.398029 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12596bac740142a99724a4bfd119bc77'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.407882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:29.419614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f13ccc79-2873-4aee-94d5-2ebb99811f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12596bac740142a99724a4bfd119bc77'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:29.429320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1340/2706] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:29 2022] DELETE /identity/v3/users/12596bac740142a99724a4bfd119bc77 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:29.906067 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:29.907647 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:29.928702 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47952608-a1f9-486a-9579-97419c4d2608 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:29.997786 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:29.998449 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:29.998724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:29.998991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] PATH_INFO: `/v3/projects/14bc2dad41104fb284535aef4befb504` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.000135 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14bc2dad41104fb284535aef4befb504)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.002078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.054266 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14bc2dad41104fb284535aef4befb504', 'request_id': 'req-c7d46866-6172-4fbf-8707-9255c79d0f56'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:30.060361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14bc2dad41104fb284535aef4befb504', 'request_id': 'req-c7d46866-6172-4fbf-8707-9255c79d0f56'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:30.066762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14bc2dad41104fb284535aef4befb504', 'request_id': 'req-c7d46866-6172-4fbf-8707-9255c79d0f56'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:30.078822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] The token cache is being invalidate because project 14bc2dad41104fb284535aef4befb504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:30.079104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:30.080197 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c7d46866-6172-4fbf-8707-9255c79d0f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:30.081680 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1367/2707] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:29 2022] DELETE /identity/v3/projects/14bc2dad41104fb284535aef4befb504 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:30.086473 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5046c-f120-4d52-a49a-37ae90f19c0c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.086757 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5046c-f120-4d52-a49a-37ae90f19c0c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.087051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5046c-f120-4d52-a49a-37ae90f19c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.112232 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9cb5046c-f120-4d52-a49a-37ae90f19c0c None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:30.113745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9cb5046c-f120-4d52-a49a-37ae90f19c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:30.179955 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1341/2708] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:30.190404 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7122a22-5842-411e-8f83-554b96084c5b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.190938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7122a22-5842-411e-8f83-554b96084c5b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.191205 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7122a22-5842-411e-8f83-554b96084c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.211560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f7122a22-5842-411e-8f83-554b96084c5b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:30.212766 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f7122a22-5842-411e-8f83-554b96084c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:30.221393 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1368/2709] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:30.225404 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5046c-f120-4d52-a49a-37ae90f19c0c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.238149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5046c-f120-4d52-a49a-37ae90f19c0c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.242983 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.243667 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.243970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.244246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.244992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:45:30.245327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.246941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.259701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1342/2710] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:45:30.264528 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7122a22-5842-411e-8f83-554b96084c5b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.276096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7122a22-5842-411e-8f83-554b96084c5b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.281382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a552d-7215-40c2-8531-5642041e033c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.281986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a552d-7215-40c2-8531-5642041e033c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.282257 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a552d-7215-40c2-8531-5642041e033c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.282521 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a552d-7215-40c2-8531-5642041e033c admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.283148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a552d-7215-40c2-8531-5642041e033c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.285041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a552d-7215-40c2-8531-5642041e033c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.290432 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:30.313115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1369/2711] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:30 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:45:30.317544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.328937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5b24e-50dd-48f0-bc3f-eeb715de858c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.387013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b3fb9-f183-4ef3-95bd-82ead44949d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.387636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b3fb9-f183-4ef3-95bd-82ead44949d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.387919 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b3fb9-f183-4ef3-95bd-82ead44949d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.388190 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b3fb9-f183-4ef3-95bd-82ead44949d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.388824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b3fb9-f183-4ef3-95bd-82ead44949d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.390774 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b3fb9-f183-4ef3-95bd-82ead44949d7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.413865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1343/2712] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:30 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:30.418014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a552d-7215-40c2-8531-5642041e033c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.428318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a552d-7215-40c2-8531-5642041e033c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.433196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf7019-f677-4a43-bfed-0c60b1828f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.433858 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf7019-f677-4a43-bfed-0c60b1828f1c admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.434128 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf7019-f677-4a43-bfed-0c60b1828f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.434418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf7019-f677-4a43-bfed-0c60b1828f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.435049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf7019-f677-4a43-bfed-0c60b1828f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.436667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf7019-f677-4a43-bfed-0c60b1828f1c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.447406 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1370/2713] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:30.451755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b3fb9-f183-4ef3-95bd-82ead44949d7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.461724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b3fb9-f183-4ef3-95bd-82ead44949d7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.467615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04979a85-6b6a-45a0-bcc6-053067860495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.468243 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04979a85-6b6a-45a0-bcc6-053067860495 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.468516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04979a85-6b6a-45a0-bcc6-053067860495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.468783 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04979a85-6b6a-45a0-bcc6-053067860495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.469466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04979a85-6b6a-45a0-bcc6-053067860495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.471048 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04979a85-6b6a-45a0-bcc6-053067860495 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.482143 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1344/2714] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:30.486552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf7019-f677-4a43-bfed-0c60b1828f1c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.496778 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf7019-f677-4a43-bfed-0c60b1828f1c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.501332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fd502-19c4-4bce-bd30-6b270546b0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.501964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fd502-19c4-4bce-bd30-6b270546b0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.502237 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fd502-19c4-4bce-bd30-6b270546b0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.502501 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fd502-19c4-4bce-bd30-6b270546b0f6 admin admin] PATH_INFO: `/v3/projects/e9288defa50d4c4081d3b2587439df10/users/d3bab3e63d3c4ae1b5862c96d9af149a/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.517016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fd502-19c4-4bce-bd30-6b270546b0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9288defa50d4c4081d3b2587439df10, user_id=d3bab3e63d3c4ae1b5862c96d9af149a, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.519696 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fd502-19c4-4bce-bd30-6b270546b0f6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.538407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1371/2715] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:30 2022] PUT /identity/v3/projects/e9288defa50d4c4081d3b2587439df10/users/d3bab3e63d3c4ae1b5862c96d9af149a/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:30.549018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9692e-d3d7-424d-b87d-1df1110cdb18 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.549220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9692e-d3d7-424d-b87d-1df1110cdb18 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.549516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9692e-d3d7-424d-b87d-1df1110cdb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.590723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3ff9692e-d3d7-424d-b87d-1df1110cdb18 None None] MFA Rules not processed for user `d3bab3e63d3c4ae1b5862c96d9af149a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:30.591960 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3ff9692e-d3d7-424d-b87d-1df1110cdb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:30.637588 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:30.663862 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1345/2716] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:30.668730 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fd502-19c4-4bce-bd30-6b270546b0f6 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.739744 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fd502-19c4-4bce-bd30-6b270546b0f6 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.745315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201edb8-a5f0-4613-a580-1b6dd1637828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.746236 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201edb8-a5f0-4613-a580-1b6dd1637828 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.746668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201edb8-a5f0-4613-a580-1b6dd1637828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.747091 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201edb8-a5f0-4613-a580-1b6dd1637828 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.748179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201edb8-a5f0-4613-a580-1b6dd1637828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.751051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201edb8-a5f0-4613-a580-1b6dd1637828 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.777869 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1372/2717] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:30 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:30.781992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9692e-d3d7-424d-b87d-1df1110cdb18 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.793018 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9692e-d3d7-424d-b87d-1df1110cdb18 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.850515 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc540ee-0c2a-4cd8-9292-0f9934263f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.851124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc540ee-0c2a-4cd8-9292-0f9934263f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.851400 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc540ee-0c2a-4cd8-9292-0f9934263f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.851668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc540ee-0c2a-4cd8-9292-0f9934263f19 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.852281 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc540ee-0c2a-4cd8-9292-0f9934263f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.854229 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc540ee-0c2a-4cd8-9292-0f9934263f19 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.879716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1346/2718] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:30 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:30.883497 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201edb8-a5f0-4613-a580-1b6dd1637828 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.894001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201edb8-a5f0-4613-a580-1b6dd1637828 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.898560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897143-b692-4150-a419-1b2dead409a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.899179 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897143-b692-4150-a419-1b2dead409a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.899455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897143-b692-4150-a419-1b2dead409a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.899724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897143-b692-4150-a419-1b2dead409a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.900356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e897143-b692-4150-a419-1b2dead409a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.902553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e897143-b692-4150-a419-1b2dead409a0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.915036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1373/2719] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:30.919901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc540ee-0c2a-4cd8-9292-0f9934263f19 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.931812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc540ee-0c2a-4cd8-9292-0f9934263f19 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:30.936410 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cb31f-fd97-47d3-853b-254730afde38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:30.937064 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cb31f-fd97-47d3-853b-254730afde38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:30.937371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cb31f-fd97-47d3-853b-254730afde38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:30.937644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cb31f-fd97-47d3-853b-254730afde38 admin admin] PATH_INFO: `/v3/projects/2f66a370c52741318cab6c85b2359df3/users/b4e9537edb1b471d812cb04b678aa5d5/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:30.952502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cb31f-fd97-47d3-853b-254730afde38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f66a370c52741318cab6c85b2359df3, user_id=b4e9537edb1b471d812cb04b678aa5d5, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:30.955219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cb31f-fd97-47d3-853b-254730afde38 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:30.969755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1347/2720] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:30 2022] PUT /identity/v3/projects/2f66a370c52741318cab6c85b2359df3/users/b4e9537edb1b471d812cb04b678aa5d5/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:30.973570 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897143-b692-4150-a419-1b2dead409a0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:30.983837 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897143-b692-4150-a419-1b2dead409a0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:31.040331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d36c4-d498-4547-990b-0a0e0dc92ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:31.041054 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d36c4-d498-4547-990b-0a0e0dc92ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:31.041591 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d36c4-d498-4547-990b-0a0e0dc92ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:31.041986 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d36c4-d498-4547-990b-0a0e0dc92ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:31.044921 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d36c4-d498-4547-990b-0a0e0dc92ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:31.045108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d36c4-d498-4547-990b-0a0e0dc92ca3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:31.056488 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1374/2721] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:31.060873 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cb31f-fd97-47d3-853b-254730afde38 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:31.071149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cb31f-fd97-47d3-853b-254730afde38 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:31.076813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:31.077502 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:31.077784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:31.078057 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] PATH_INFO: `/v3/projects/2f66a370c52741318cab6c85b2359df3/users/b4e9537edb1b471d812cb04b678aa5d5/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:31.080045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f66a370c52741318cab6c85b2359df3, user_id=b4e9537edb1b471d812cb04b678aa5d5, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:31.082812 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:31.094200 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1348/2722] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:31 2022] PUT /identity/v3/projects/2f66a370c52741318cab6c85b2359df3/users/b4e9537edb1b471d812cb04b678aa5d5/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:31.105426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf7405-643b-4b67-bad0-12c6435f959a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:31.105754 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf7405-643b-4b67-bad0-12c6435f959a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:31.106001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf7405-643b-4b67-bad0-12c6435f959a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:31.133876 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3bdf7405-643b-4b67-bad0-12c6435f959a None None] MFA Rules not processed for user `b4e9537edb1b471d812cb04b678aa5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:31.135247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3bdf7405-643b-4b67-bad0-12c6435f959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:31.145414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:31.214559 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1375/2723] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:31.222244 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:31.224156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:31.293045 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afff081-9209-4ad7-a47e-9533105dd1b9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:31.297087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394825d2-ac2d-4f56-ab02-5161df32e462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:31.297945 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394825d2-ac2d-4f56-ab02-5161df32e462 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:31.298395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394825d2-ac2d-4f56-ab02-5161df32e462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:31.298814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394825d2-ac2d-4f56-ab02-5161df32e462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:31.311492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394825d2-ac2d-4f56-ab02-5161df32e462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:31.313816 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394825d2-ac2d-4f56-ab02-5161df32e462 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:31.330215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1349/2724] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:31.420471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf7405-643b-4b67-bad0-12c6435f959a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:31.430865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf7405-643b-4b67-bad0-12c6435f959a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:31.436763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63b5dd-d3d3-49bc-86ec-a23205011961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:31.437407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63b5dd-d3d3-49bc-86ec-a23205011961 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:31.437688 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63b5dd-d3d3-49bc-86ec-a23205011961 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:31.437960 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63b5dd-d3d3-49bc-86ec-a23205011961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:31.527538 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63b5dd-d3d3-49bc-86ec-a23205011961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:31.529078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63b5dd-d3d3-49bc-86ec-a23205011961 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:31.539254 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1376/2725] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:41.323574 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:41.534097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=52 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:41.938975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394825d2-ac2d-4f56-ab02-5161df32e462 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:42.047016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394825d2-ac2d-4f56-ab02-5161df32e462 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:42.051618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:42.052259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:42.052540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:42.052819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] PATH_INFO: `/v3/users/d3bab3e63d3c4ae1b5862c96d9af149a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:42.053981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3bab3e63d3c4ae1b5862c96d9af149a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:42.055892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:42.103898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3bab3e63d3c4ae1b5862c96d9af149a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.111068 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3bab3e63d3c4ae1b5862c96d9af149a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.111716 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3bab3e63d3c4ae1b5862c96d9af149a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.120605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3bab3e63d3c4ae1b5862c96d9af149a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.132357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3bab3e63d3c4ae1b5862c96d9af149a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.138957 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1350/2726] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:41 2022] DELETE /identity/v3/users/d3bab3e63d3c4ae1b5862c96d9af149a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:42.143514 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381069-cd68-4fa5-9c15-8f994017e4b7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:42.143798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381069-cd68-4fa5-9c15-8f994017e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:42.144073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381069-cd68-4fa5-9c15-8f994017e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:42.193918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-88381069-cd68-4fa5-9c15-8f994017e4b7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:42.195125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-88381069-cd68-4fa5-9c15-8f994017e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:42.254299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1377/2727] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:45:42.263746 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:42.265150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:42.307991 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:42.343790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62e1e0-bfd9-43eb-b5eb-10f7d701e4a3 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:42.347461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09afc79d-61c7-4311-aa6c-fb6071e352c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:42.348065 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afc79d-61c7-4311-aa6c-fb6071e352c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:42.348337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afc79d-61c7-4311-aa6c-fb6071e352c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:42.348600 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afc79d-61c7-4311-aa6c-fb6071e352c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:42.357641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afc79d-61c7-4311-aa6c-fb6071e352c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:42.359050 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afc79d-61c7-4311-aa6c-fb6071e352c1 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:42.368584 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1351/2728] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:42.663333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88381069-cd68-4fa5-9c15-8f994017e4b7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:42.664798 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-88381069-cd68-4fa5-9c15-8f994017e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:42.678344 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88381069-cd68-4fa5-9c15-8f994017e4b7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:42.683503 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:42.684125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:42.684402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:42.684667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] PATH_INFO: `/v3/projects/e9288defa50d4c4081d3b2587439df10` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:42.685821 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9288defa50d4c4081d3b2587439df10)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:42.687761 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:42.735319 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9288defa50d4c4081d3b2587439df10', 'request_id': 'req-f58b48df-f33a-4eb0-9c36-213d92af16fb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.741051 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9288defa50d4c4081d3b2587439df10', 'request_id': 'req-f58b48df-f33a-4eb0-9c36-213d92af16fb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.747667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9288defa50d4c4081d3b2587439df10', 'request_id': 'req-f58b48df-f33a-4eb0-9c36-213d92af16fb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.761075 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] The token cache is being invalidate because project e9288defa50d4c4081d3b2587439df10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:42.761400 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.762009 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.763575 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1378/2729] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:42 2022] DELETE /identity/v3/projects/e9288defa50d4c4081d3b2587439df10 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:42.767622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09afc79d-61c7-4311-aa6c-fb6071e352c1 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:42.777097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09afc79d-61c7-4311-aa6c-fb6071e352c1 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:42.831973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:42.832514 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:42.832786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:42.833051 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] PATH_INFO: `/v3/users/b4e9537edb1b471d812cb04b678aa5d5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:42.834166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e9537edb1b471d812cb04b678aa5d5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:42.836109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:42.889239 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e9537edb1b471d812cb04b678aa5d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.896630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e9537edb1b471d812cb04b678aa5d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.897323 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e9537edb1b471d812cb04b678aa5d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.904880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e9537edb1b471d812cb04b678aa5d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.917474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8cc29eb8-012d-46f6-b989-671a541fbf83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e9537edb1b471d812cb04b678aa5d5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:42.923814 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1352/2730] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:45:42 2022] DELETE /identity/v3/users/b4e9537edb1b471d812cb04b678aa5d5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:43.210035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:43.211572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:43.226092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b48df-f33a-4eb0-9c36-213d92af16fb admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:43.281399 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:43.288782 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:43.289629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.289982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.290287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] PATH_INFO: `/v3/projects/2f66a370c52741318cab6c85b2359df3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.291562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f66a370c52741318cab6c85b2359df3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:43.293673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:43.338929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f66a370c52741318cab6c85b2359df3', 'request_id': 'req-ab344d91-eb64-4fce-9788-6a330fa878c5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:43.344299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f66a370c52741318cab6c85b2359df3', 'request_id': 'req-ab344d91-eb64-4fce-9788-6a330fa878c5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:43.350263 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f66a370c52741318cab6c85b2359df3', 'request_id': 'req-ab344d91-eb64-4fce-9788-6a330fa878c5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:43.362291 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] The token cache is being invalidate because project 2f66a370c52741318cab6c85b2359df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:45:43.362576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:43.363452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ab344d91-eb64-4fce-9788-6a330fa878c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:45:43.364916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1379/2731] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:45:43 2022] DELETE /identity/v3/projects/2f66a370c52741318cab6c85b2359df3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:43.371074 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab528af-a6c8-437d-b17b-fe8d6524b6ca None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.371563 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab528af-a6c8-437d-b17b-fe8d6524b6ca None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.371661 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab528af-a6c8-437d-b17b-fe8d6524b6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.422810 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-aab528af-a6c8-437d-b17b-fe8d6524b6ca None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:43.424011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-aab528af-a6c8-437d-b17b-fe8d6524b6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:43.436889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:43.483163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1353/2732] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:43.491206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ab17-7339-429e-8e40-32834fa9caeb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.491487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ab17-7339-429e-8e40-32834fa9caeb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.491759 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ab17-7339-429e-8e40-32834fa9caeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.512532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-bca5ab17-7339-429e-8e40-32834fa9caeb None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:43.516463 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bca5ab17-7339-429e-8e40-32834fa9caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:43.522096 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1380/2733] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:43.526301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab528af-a6c8-437d-b17b-fe8d6524b6ca None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:43.536004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab528af-a6c8-437d-b17b-fe8d6524b6ca None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:43.540461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:43.541053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.541357 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.541625 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.542259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:45:43.542554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:43.544156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:43.555566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1354/2734] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:45:43.560356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5ab17-7339-429e-8e40-32834fa9caeb None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:43.569532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5ab17-7339-429e-8e40-32834fa9caeb None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:43.574352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53032ccd-aef5-49d6-963f-6e2febd455a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:43.575015 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53032ccd-aef5-49d6-963f-6e2febd455a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.575295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53032ccd-aef5-49d6-963f-6e2febd455a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.575560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53032ccd-aef5-49d6-963f-6e2febd455a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.576194 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53032ccd-aef5-49d6-963f-6e2febd455a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:43.578677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53032ccd-aef5-49d6-963f-6e2febd455a5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:43.611104 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1381/2735] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:43 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:43.612308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:43.623034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b47e4-844b-4c05-881b-3f6eb71d87b9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:43.681010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702c183-62de-4f82-ba52-5aa84d500b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:43.681717 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702c183-62de-4f82-ba52-5aa84d500b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.682044 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702c183-62de-4f82-ba52-5aa84d500b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.682835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702c183-62de-4f82-ba52-5aa84d500b45 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.683484 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702c183-62de-4f82-ba52-5aa84d500b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:43.685430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702c183-62de-4f82-ba52-5aa84d500b45 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:43.706490 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1355/2736] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:43 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:43.710799 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53032ccd-aef5-49d6-963f-6e2febd455a5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:43.724288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53032ccd-aef5-49d6-963f-6e2febd455a5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:43.730471 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4dc62-bdb2-4097-9442-603e255ced02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:43.731429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4dc62-bdb2-4097-9442-603e255ced02 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.731894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4dc62-bdb2-4097-9442-603e255ced02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.732337 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4dc62-bdb2-4097-9442-603e255ced02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.733494 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4dc62-bdb2-4097-9442-603e255ced02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:43.736252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4dc62-bdb2-4097-9442-603e255ced02 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:43.751454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1382/2737] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:43.755494 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702c183-62de-4f82-ba52-5aa84d500b45 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:43.767388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702c183-62de-4f82-ba52-5aa84d500b45 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:43.772787 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc54c7-d293-4ac9-8494-737d911ed9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:43.773480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc54c7-d293-4ac9-8494-737d911ed9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.773762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc54c7-d293-4ac9-8494-737d911ed9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.774034 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc54c7-d293-4ac9-8494-737d911ed9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.774773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc54c7-d293-4ac9-8494-737d911ed9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:43.776386 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc54c7-d293-4ac9-8494-737d911ed9f8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:43.787592 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1356/2738] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:43.792289 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4dc62-bdb2-4097-9442-603e255ced02 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:43.802535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4dc62-bdb2-4097-9442-603e255ced02 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:43.807777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17070cc4-9eee-40b5-ae03-b889756f9f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:43.808417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070cc4-9eee-40b5-ae03-b889756f9f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.808690 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070cc4-9eee-40b5-ae03-b889756f9f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.808956 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070cc4-9eee-40b5-ae03-b889756f9f44 admin admin] PATH_INFO: `/v3/projects/63dcd88b22a146149c022f72dd7b825c/users/edd872164c8a428da8c60a3baf818074/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.823081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17070cc4-9eee-40b5-ae03-b889756f9f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63dcd88b22a146149c022f72dd7b825c, user_id=edd872164c8a428da8c60a3baf818074, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:43.825739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17070cc4-9eee-40b5-ae03-b889756f9f44 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:43.838072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1383/2739] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:43 2022] PUT /identity/v3/projects/63dcd88b22a146149c022f72dd7b825c/users/edd872164c8a428da8c60a3baf818074/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:43.848352 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f953a-b46b-458d-974b-be6843217754 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:43.848597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f953a-b46b-458d-974b-be6843217754 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:43.848869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f953a-b46b-458d-974b-be6843217754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:43.878912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-076f953a-b46b-458d-974b-be6843217754 None None] MFA Rules not processed for user `edd872164c8a428da8c60a3baf818074`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:43.880134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-076f953a-b46b-458d-974b-be6843217754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:43.944006 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1357/2740] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:43.948587 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17070cc4-9eee-40b5-ae03-b889756f9f44 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:44.012331 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17070cc4-9eee-40b5-ae03-b889756f9f44 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:44.016922 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7965fe9-bf80-4a93-8258-210708fb2a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:44.017551 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7965fe9-bf80-4a93-8258-210708fb2a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:44.017827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7965fe9-bf80-4a93-8258-210708fb2a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:44.018095 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7965fe9-bf80-4a93-8258-210708fb2a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:44.018719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7965fe9-bf80-4a93-8258-210708fb2a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:44.020600 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7965fe9-bf80-4a93-8258-210708fb2a45 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:44.053894 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1384/2741] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:45:43 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:44.057871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f953a-b46b-458d-974b-be6843217754 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:44.068489 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f953a-b46b-458d-974b-be6843217754 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:44.079686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:44.126296 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c53cb-f07c-4b2b-81a5-caff1424a69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:44.126971 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c53cb-f07c-4b2b-81a5-caff1424a69e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:44.127347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c53cb-f07c-4b2b-81a5-caff1424a69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:44.127680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c53cb-f07c-4b2b-81a5-caff1424a69e admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:44.128308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c53cb-f07c-4b2b-81a5-caff1424a69e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:44.130267 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c53cb-f07c-4b2b-81a5-caff1424a69e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:44.156992 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1358/2742] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:45:44 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:45:44.158420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7965fe9-bf80-4a93-8258-210708fb2a45 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:44.169789 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7965fe9-bf80-4a93-8258-210708fb2a45 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:44.174345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31357067-252d-4972-8b75-0e92d43a73d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:44.174929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31357067-252d-4972-8b75-0e92d43a73d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:44.175316 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31357067-252d-4972-8b75-0e92d43a73d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:44.175628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31357067-252d-4972-8b75-0e92d43a73d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:44.176258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31357067-252d-4972-8b75-0e92d43a73d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:44.177861 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31357067-252d-4972-8b75-0e92d43a73d9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:44.179850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:44.189935 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1385/2743] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:44.193805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c53cb-f07c-4b2b-81a5-caff1424a69e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:44.203696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c53cb-f07c-4b2b-81a5-caff1424a69e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:44.208497 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f7f7f-0ea2-498d-b164-6f79483a4198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:44.209145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f7f7f-0ea2-498d-b164-6f79483a4198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:44.209479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f7f7f-0ea2-498d-b164-6f79483a4198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:44.209750 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f7f7f-0ea2-498d-b164-6f79483a4198 admin admin] PATH_INFO: `/v3/projects/e50f01f99a3347e5b53aef6d7a10a3cd/users/a13ed43352734ff991cb0e27a005871a/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:44.224961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f7f7f-0ea2-498d-b164-6f79483a4198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50f01f99a3347e5b53aef6d7a10a3cd, user_id=a13ed43352734ff991cb0e27a005871a, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:44.227611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f7f7f-0ea2-498d-b164-6f79483a4198 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:44.242745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1359/2744] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:44 2022] PUT /identity/v3/projects/e50f01f99a3347e5b53aef6d7a10a3cd/users/a13ed43352734ff991cb0e27a005871a/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:44.250781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31357067-252d-4972-8b75-0e92d43a73d9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:44.256739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31357067-252d-4972-8b75-0e92d43a73d9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:44.315099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799fabe-8809-4bcf-a7ae-9f364169cb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:44.315701 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799fabe-8809-4bcf-a7ae-9f364169cb84 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:44.315978 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799fabe-8809-4bcf-a7ae-9f364169cb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:44.316246 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799fabe-8809-4bcf-a7ae-9f364169cb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:44.316877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799fabe-8809-4bcf-a7ae-9f364169cb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:44.318498 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799fabe-8809-4bcf-a7ae-9f364169cb84 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:44.327941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1386/2745] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:45:44.332265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f7f7f-0ea2-498d-b164-6f79483a4198 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:44.342967 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f7f7f-0ea2-498d-b164-6f79483a4198 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:44.347819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:44.348493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:44.348767 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:44.349063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] PATH_INFO: `/v3/projects/e50f01f99a3347e5b53aef6d7a10a3cd/users/a13ed43352734ff991cb0e27a005871a/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:44.350981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50f01f99a3347e5b53aef6d7a10a3cd, user_id=a13ed43352734ff991cb0e27a005871a, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:44.353641 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:44.364712 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1360/2746] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:45:44 2022] PUT /identity/v3/projects/e50f01f99a3347e5b53aef6d7a10a3cd/users/a13ed43352734ff991cb0e27a005871a/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:45:44.375765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3a689-6401-46c3-9069-b1486a53e3b7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:44.376050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3a689-6401-46c3-9069-b1486a53e3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:44.376320 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3a689-6401-46c3-9069-b1486a53e3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:44.408803 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-59a3a689-6401-46c3-9069-b1486a53e3b7 None None] MFA Rules not processed for user `a13ed43352734ff991cb0e27a005871a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:45:44.410053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-59a3a689-6401-46c3-9069-b1486a53e3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:44.477867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1387/2747] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:45:44.488835 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:44.490596 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:44.550803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a7373-f070-472c-bd08-26c156b87f5e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:44.554413 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:44.555013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:44.555288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:44.555556 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:44.611859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:44.613604 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:44.622907 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1361/2748] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:45.930691 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3a689-6401-46c3-9069-b1486a53e3b7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:45.932088 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-59a3a689-6401-46c3-9069-b1486a53e3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:46.024359 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:45:46.029697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3a689-6401-46c3-9069-b1486a53e3b7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:46.037034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:46.037709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:46.037984 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:46.038247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:46.039995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:46.052572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:46.054313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:46.066572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1388/2749] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:50.308121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:45:50.309582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:45:50.322101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23598c9c-b982-4698-9b96-59c3ba3b3eff service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:45:50.326370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d04a83-8885-48d1-91b2-e4f29fa03fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:45:50.326965 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04a83-8885-48d1-91b2-e4f29fa03fef service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:45:50.327237 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04a83-8885-48d1-91b2-e4f29fa03fef service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:45:50.327505 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04a83-8885-48d1-91b2-e4f29fa03fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:45:50.336434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d04a83-8885-48d1-91b2-e4f29fa03fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:45:50.337917 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d04a83-8885-48d1-91b2-e4f29fa03fef service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:45:50.348148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1362/2750] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:45:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:45:56.060762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:00.343170 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:20.296338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:20.297942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:20.380770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40928e-b6f6-4559-80d1-329a0774bc37 service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:20.388013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:20.388721 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:20.388998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:20.389292 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] PATH_INFO: `/v3/users/a4e2ef58773a46b998162cb555acdbda` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:20.390440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e2ef58773a46b998162cb555acdbda)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:20.392391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:20.442739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e2ef58773a46b998162cb555acdbda'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:20.450930 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e2ef58773a46b998162cb555acdbda'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:20.451586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e2ef58773a46b998162cb555acdbda'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:20.463774 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e2ef58773a46b998162cb555acdbda'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:20.464430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e2ef58773a46b998162cb555acdbda'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:20.476720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1389/2751] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:46:20 2022] DELETE /identity/v3/users/a4e2ef58773a46b998162cb555acdbda => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:20.481724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2bd1a-726b-4735-bfcb-5582a7c3b233 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:20.482209 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2bd1a-726b-4735-bfcb-5582a7c3b233 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:20.482308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2bd1a-726b-4735-bfcb-5582a7c3b233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:20.503023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f7a2bd1a-726b-4735-bfcb-5582a7c3b233 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:20.504217 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f7a2bd1a-726b-4735-bfcb-5582a7c3b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:20.565160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1363/2752] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:20.575777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:20.577486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:20.661897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70913c-57a4-4d0c-9e9e-392ee3d6e159 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:20.666195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0405c8-a32f-4180-8a01-aea0ff85c21b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:20.666815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0405c8-a32f-4180-8a01-aea0ff85c21b service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:20.667090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0405c8-a32f-4180-8a01-aea0ff85c21b service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:20.667358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0405c8-a32f-4180-8a01-aea0ff85c21b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:20.677177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0405c8-a32f-4180-8a01-aea0ff85c21b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:20.678676 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0405c8-a32f-4180-8a01-aea0ff85c21b service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:20.690448 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1390/2753] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:21.002333 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2bd1a-726b-4735-bfcb-5582a7c3b233 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:21.005043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f7a2bd1a-726b-4735-bfcb-5582a7c3b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:21.018944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2bd1a-726b-4735-bfcb-5582a7c3b233 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:21.026108 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:21.026852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.027025 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.027321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] PATH_INFO: `/v3/projects/2cc911cd994a49e99bae085ad2f6defa` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.028429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc911cd994a49e99bae085ad2f6defa)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:21.030423 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:21.080702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc911cd994a49e99bae085ad2f6defa', 'request_id': 'req-c8a68d74-9481-423d-abd3-baf6d2cbef11'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:21.086500 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc911cd994a49e99bae085ad2f6defa', 'request_id': 'req-c8a68d74-9481-423d-abd3-baf6d2cbef11'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:21.093603 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc911cd994a49e99bae085ad2f6defa', 'request_id': 'req-c8a68d74-9481-423d-abd3-baf6d2cbef11'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:21.108299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] The token cache is being invalidate because project 2cc911cd994a49e99bae085ad2f6defa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:46:21.108581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:21.109193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c8a68d74-9481-423d-abd3-baf6d2cbef11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:21.110976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1364/2754] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:46:21 2022] DELETE /identity/v3/projects/2cc911cd994a49e99bae085ad2f6defa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:21.116079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80b1a0-ea9d-4820-99b0-d758046af92e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.116352 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80b1a0-ea9d-4820-99b0-d758046af92e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.116619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80b1a0-ea9d-4820-99b0-d758046af92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.124156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:21.139561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fa80b1a0-ea9d-4820-99b0-d758046af92e None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:21.140777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fa80b1a0-ea9d-4820-99b0-d758046af92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:21.209925 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1391/2755] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:21.218987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a874418-6497-4868-9ef1-3b6f946b7274 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.219270 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a874418-6497-4868-9ef1-3b6f946b7274 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.219566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a874418-6497-4868-9ef1-3b6f946b7274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.240332 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5a874418-6497-4868-9ef1-3b6f946b7274 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:21.241689 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5a874418-6497-4868-9ef1-3b6f946b7274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:21.250411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1365/2756] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:21.255588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80b1a0-ea9d-4820-99b0-d758046af92e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:21.272851 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80b1a0-ea9d-4820-99b0-d758046af92e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:21.278664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:21.279288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.279563 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.279827 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.280470 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:46:21.280852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:21.282396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:21.295027 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1392/2757] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:46:21.299726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a874418-6497-4868-9ef1-3b6f946b7274 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:21.309398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a874418-6497-4868-9ef1-3b6f946b7274 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:21.314389 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d3d3c-6208-4ed4-a6f2-d8a114e3216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:21.314993 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d3d3c-6208-4ed4-a6f2-d8a114e3216a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.315293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d3d3c-6208-4ed4-a6f2-d8a114e3216a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.315565 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d3d3c-6208-4ed4-a6f2-d8a114e3216a admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.316252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d3d3c-6208-4ed4-a6f2-d8a114e3216a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:21.318385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d3d3c-6208-4ed4-a6f2-d8a114e3216a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:21.347927 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1366/2758] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:46:21 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:21.352210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:21.365047 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb60bf8-61cb-4d99-957c-86879076b5e1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:21.421974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d74209-598a-4444-bd76-2713731ae1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:21.422589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74209-598a-4444-bd76-2713731ae1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.422865 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74209-598a-4444-bd76-2713731ae1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.423624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74209-598a-4444-bd76-2713731ae1f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.423751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d74209-598a-4444-bd76-2713731ae1f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:21.425751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d74209-598a-4444-bd76-2713731ae1f9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:21.455695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1393/2759] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:46:21 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:46:21.457480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d3d3c-6208-4ed4-a6f2-d8a114e3216a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:21.469196 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d3d3c-6208-4ed4-a6f2-d8a114e3216a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:21.473996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266f3c0-449d-4603-9bf0-5dc3fcd012d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:21.474594 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266f3c0-449d-4603-9bf0-5dc3fcd012d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.474888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266f3c0-449d-4603-9bf0-5dc3fcd012d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.475155 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266f3c0-449d-4603-9bf0-5dc3fcd012d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.475820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266f3c0-449d-4603-9bf0-5dc3fcd012d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:21.477432 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266f3c0-449d-4603-9bf0-5dc3fcd012d8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:21.487509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1367/2760] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:21.492312 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d74209-598a-4444-bd76-2713731ae1f9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:21.503443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d74209-598a-4444-bd76-2713731ae1f9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:21.508206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde77f4b-d8ff-4a54-a5cd-7a41f9420f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:21.508793 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde77f4b-d8ff-4a54-a5cd-7a41f9420f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.509071 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde77f4b-d8ff-4a54-a5cd-7a41f9420f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.509870 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde77f4b-d8ff-4a54-a5cd-7a41f9420f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.510505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde77f4b-d8ff-4a54-a5cd-7a41f9420f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:21.512092 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde77f4b-d8ff-4a54-a5cd-7a41f9420f76 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:21.522784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1394/2761] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:21.527875 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266f3c0-449d-4603-9bf0-5dc3fcd012d8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:21.538043 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266f3c0-449d-4603-9bf0-5dc3fcd012d8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:21.542870 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:21.543527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.543799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.544063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] PATH_INFO: `/v3/projects/b48a8b2af6ad4f59a6b5ee3fcddd66a8/users/d7bd571bcb71452299829b7dcda8f944/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.561674 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48a8b2af6ad4f59a6b5ee3fcddd66a8, user_id=d7bd571bcb71452299829b7dcda8f944, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:21.564354 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:21.579089 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1368/2762] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:46:21 2022] PUT /identity/v3/projects/b48a8b2af6ad4f59a6b5ee3fcddd66a8/users/d7bd571bcb71452299829b7dcda8f944/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:21.589856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e4139-7122-48d2-a6d3-6d4277872e81 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.590136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e4139-7122-48d2-a6d3-6d4277872e81 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.590398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e4139-7122-48d2-a6d3-6d4277872e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.619414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-955e4139-7122-48d2-a6d3-6d4277872e81 None None] MFA Rules not processed for user `d7bd571bcb71452299829b7dcda8f944`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:21.620642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-955e4139-7122-48d2-a6d3-6d4277872e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:21.682045 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:21.689251 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1395/2763] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:21.701682 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:21.703336 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:21.724684 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:21.764261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86754bcb-8b34-46df-bfd0-f8a31989066a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:21.768001 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e560ee9-aa9f-4e2f-b757-0904a646ed4a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce851cf0509947e783dbd5b4809bc897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:21.768618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e560ee9-aa9f-4e2f-b757-0904a646ed4a service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:21.768898 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e560ee9-aa9f-4e2f-b757-0904a646ed4a service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:21.769174 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e560ee9-aa9f-4e2f-b757-0904a646ed4a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:21.777805 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e560ee9-aa9f-4e2f-b757-0904a646ed4a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:21.779252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e560ee9-aa9f-4e2f-b757-0904a646ed4a service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:21.788340 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1369/2764] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:28.827805 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e4139-7122-48d2-a6d3-6d4277872e81 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:28.829258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-955e4139-7122-48d2-a6d3-6d4277872e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:28.899513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e4139-7122-48d2-a6d3-6d4277872e81 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:28.903982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:28.904621 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:28.904892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:28.905653 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] PATH_INFO: `/v3/users/edd872164c8a428da8c60a3baf818074` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:28.907221 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd872164c8a428da8c60a3baf818074)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:28.909566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:28.957100 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd872164c8a428da8c60a3baf818074'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:28.963668 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd872164c8a428da8c60a3baf818074'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:28.964358 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd872164c8a428da8c60a3baf818074'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:28.975974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd872164c8a428da8c60a3baf818074'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:28.976628 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd872164c8a428da8c60a3baf818074'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:28.990190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1396/2765] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:46:28 2022] DELETE /identity/v3/users/edd872164c8a428da8c60a3baf818074 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:28.995455 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61587f1-cf04-44c2-8d55-84f4b4be668b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:28.995931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61587f1-cf04-44c2-8d55-84f4b4be668b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:28.996037 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61587f1-cf04-44c2-8d55-84f4b4be668b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:29.014172 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c61587f1-cf04-44c2-8d55-84f4b4be668b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:29.015527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c61587f1-cf04-44c2-8d55-84f4b4be668b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:29.076104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1370/2766] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:46:29.086030 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:29.087961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:29.144532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd815a3-f4db-4eb7-905b-cb076566b203 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:29.148271 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd463b3-70d6-4554-9708-26770275142b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:29.148882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd463b3-70d6-4554-9708-26770275142b service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:29.149160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd463b3-70d6-4554-9708-26770275142b service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:29.149943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd463b3-70d6-4554-9708-26770275142b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:29.161644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd463b3-70d6-4554-9708-26770275142b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:29.163120 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd463b3-70d6-4554-9708-26770275142b service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:29.174644 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1397/2767] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:29.458429 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61587f1-cf04-44c2-8d55-84f4b4be668b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:29.460014 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c61587f1-cf04-44c2-8d55-84f4b4be668b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:29.473422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61587f1-cf04-44c2-8d55-84f4b4be668b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:29.478085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:29.478721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:29.479023 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:29.479288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] PATH_INFO: `/v3/projects/63dcd88b22a146149c022f72dd7b825c` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:29.480427 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=63dcd88b22a146149c022f72dd7b825c)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:29.483308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:29.531293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63dcd88b22a146149c022f72dd7b825c', 'request_id': 'req-ebaf17f5-e272-4177-97ff-01c0b47c22fc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.536655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63dcd88b22a146149c022f72dd7b825c', 'request_id': 'req-ebaf17f5-e272-4177-97ff-01c0b47c22fc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.542803 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63dcd88b22a146149c022f72dd7b825c', 'request_id': 'req-ebaf17f5-e272-4177-97ff-01c0b47c22fc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.555075 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] The token cache is being invalidate because project 63dcd88b22a146149c022f72dd7b825c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:46:29.555620 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.556398 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.558387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1371/2768] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:46:29 2022] DELETE /identity/v3/projects/63dcd88b22a146149c022f72dd7b825c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:29.563284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd463b3-70d6-4554-9708-26770275142b service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:29.573446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd463b3-70d6-4554-9708-26770275142b service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:29.593556 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:29.634738 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:29.635361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:29.635633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:29.635897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] PATH_INFO: `/v3/users/a13ed43352734ff991cb0e27a005871a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:29.636995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13ed43352734ff991cb0e27a005871a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:29.638997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:29.702216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13ed43352734ff991cb0e27a005871a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.709415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13ed43352734ff991cb0e27a005871a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.710032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13ed43352734ff991cb0e27a005871a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.722818 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13ed43352734ff991cb0e27a005871a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.723482 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ba96ddce-d5af-4731-af16-05f1fb752522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13ed43352734ff991cb0e27a005871a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:29.736193 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1398/2769] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:46:29 2022] DELETE /identity/v3/users/a13ed43352734ff991cb0e27a005871a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:30.031382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.034002 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:30.051821 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf17f5-e272-4177-97ff-01c0b47c22fc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.104265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:30.119922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.120566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.120840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.121107 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] PATH_INFO: `/v3/projects/e50f01f99a3347e5b53aef6d7a10a3cd` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.122334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50f01f99a3347e5b53aef6d7a10a3cd)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.124332 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.176337 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50f01f99a3347e5b53aef6d7a10a3cd', 'request_id': 'req-ab3ed04f-ce74-48e9-a541-8161282c43c3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:30.182533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50f01f99a3347e5b53aef6d7a10a3cd', 'request_id': 'req-ab3ed04f-ce74-48e9-a541-8161282c43c3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:30.189125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50f01f99a3347e5b53aef6d7a10a3cd', 'request_id': 'req-ab3ed04f-ce74-48e9-a541-8161282c43c3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:30.201944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] The token cache is being invalidate because project e50f01f99a3347e5b53aef6d7a10a3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:46:30.202231 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:30.202841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ab3ed04f-ce74-48e9-a541-8161282c43c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:30.211448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1372/2770] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:46:30 2022] DELETE /identity/v3/projects/e50f01f99a3347e5b53aef6d7a10a3cd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:30.216815 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74248d6-d06a-4356-ab4f-f7e1cb3a1e58 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.217315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74248d6-d06a-4356-ab4f-f7e1cb3a1e58 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.217445 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74248d6-d06a-4356-ab4f-f7e1cb3a1e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.238207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a74248d6-d06a-4356-ab4f-f7e1cb3a1e58 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:30.239443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a74248d6-d06a-4356-ab4f-f7e1cb3a1e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:30.301371 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1399/2771] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:30.309976 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea5d1-182d-4384-a4f6-d70546d5e903 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.310264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea5d1-182d-4384-a4f6-d70546d5e903 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.310537 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea5d1-182d-4384-a4f6-d70546d5e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.330753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e3eea5d1-182d-4384-a4f6-d70546d5e903 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:30.332087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e3eea5d1-182d-4384-a4f6-d70546d5e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:30.342387 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1373/2772] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:30.347011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74248d6-d06a-4356-ab4f-f7e1cb3a1e58 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.357856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74248d6-d06a-4356-ab4f-f7e1cb3a1e58 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.363240 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.363909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.364210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.364450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.365156 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:46:30.365552 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.367039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.382502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1400/2773] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:46:30.386861 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eea5d1-182d-4384-a4f6-d70546d5e903 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.396756 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eea5d1-182d-4384-a4f6-d70546d5e903 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.401509 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cf2d3-767b-4bb1-ae7f-c47fbbb68dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.402112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf2d3-767b-4bb1-ae7f-c47fbbb68dbc admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.402388 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf2d3-767b-4bb1-ae7f-c47fbbb68dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.402656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf2d3-767b-4bb1-ae7f-c47fbbb68dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.403283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cf2d3-767b-4bb1-ae7f-c47fbbb68dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.405184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cf2d3-767b-4bb1-ae7f-c47fbbb68dbc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.434950 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1374/2774] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:46:30 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:30.439121 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.449590 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff624c-08dc-4c93-9d76-54313a3a5ece admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.475967 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:30.511739 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36ddb7-0a90-4494-b64f-ec26e2f0d603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.512345 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36ddb7-0a90-4494-b64f-ec26e2f0d603 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.512619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36ddb7-0a90-4494-b64f-ec26e2f0d603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.512887 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36ddb7-0a90-4494-b64f-ec26e2f0d603 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.513566 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36ddb7-0a90-4494-b64f-ec26e2f0d603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.515477 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36ddb7-0a90-4494-b64f-ec26e2f0d603 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.537033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1401/2775] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:46:30 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:30.541802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cf2d3-767b-4bb1-ae7f-c47fbbb68dbc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.551831 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cf2d3-767b-4bb1-ae7f-c47fbbb68dbc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.556456 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c7d30-e9b4-43a7-bc3b-7bd72f22c0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.557112 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c7d30-e9b4-43a7-bc3b-7bd72f22c0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.557341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c7d30-e9b4-43a7-bc3b-7bd72f22c0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.557678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c7d30-e9b4-43a7-bc3b-7bd72f22c0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.558223 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c7d30-e9b4-43a7-bc3b-7bd72f22c0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.559799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c7d30-e9b4-43a7-bc3b-7bd72f22c0f4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.569938 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1375/2776] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:46:30.573852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36ddb7-0a90-4494-b64f-ec26e2f0d603 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.585310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36ddb7-0a90-4494-b64f-ec26e2f0d603 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.590492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3a31c-a087-4930-8ad6-09419f8cd9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.591116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3a31c-a087-4930-8ad6-09419f8cd9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.591395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3a31c-a087-4930-8ad6-09419f8cd9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.591661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3a31c-a087-4930-8ad6-09419f8cd9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.592299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3a31c-a087-4930-8ad6-09419f8cd9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.593950 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3a31c-a087-4930-8ad6-09419f8cd9b3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.605062 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1402/2777] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:30.609321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c7d30-e9b4-43a7-bc3b-7bd72f22c0f4 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.619659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c7d30-e9b4-43a7-bc3b-7bd72f22c0f4 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.624279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e75d1-d518-47e5-9ad5-84ae057d79ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.624921 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e75d1-d518-47e5-9ad5-84ae057d79ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.625367 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e75d1-d518-47e5-9ad5-84ae057d79ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.625648 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e75d1-d518-47e5-9ad5-84ae057d79ce admin admin] PATH_INFO: `/v3/projects/35542546f3d74527a69742301c2085e3/users/bd861479c1f5439bb4748cd7a4545e9e/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.649951 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e75d1-d518-47e5-9ad5-84ae057d79ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=35542546f3d74527a69742301c2085e3, user_id=bd861479c1f5439bb4748cd7a4545e9e, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.652602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e75d1-d518-47e5-9ad5-84ae057d79ce admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.666087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1376/2778] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:46:30 2022] PUT /identity/v3/projects/35542546f3d74527a69742301c2085e3/users/bd861479c1f5439bb4748cd7a4545e9e/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:30.677168 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa5e46-16e6-4258-9375-d67fabe7cccc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.677469 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa5e46-16e6-4258-9375-d67fabe7cccc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.677733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa5e46-16e6-4258-9375-d67fabe7cccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.704822 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b2fa5e46-16e6-4258-9375-d67fabe7cccc None None] MFA Rules not processed for user `bd861479c1f5439bb4748cd7a4545e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:30.706050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b2fa5e46-16e6-4258-9375-d67fabe7cccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:30.772233 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1403/2779] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:30.777282 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e75d1-d518-47e5-9ad5-84ae057d79ce admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.848063 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e75d1-d518-47e5-9ad5-84ae057d79ce admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.853339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5eb0a-fa7e-4585-916e-8acf6e613e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.853943 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5eb0a-fa7e-4585-916e-8acf6e613e66 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.854215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5eb0a-fa7e-4585-916e-8acf6e613e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.854479 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5eb0a-fa7e-4585-916e-8acf6e613e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.855128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5eb0a-fa7e-4585-916e-8acf6e613e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.857010 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5eb0a-fa7e-4585-916e-8acf6e613e66 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.877524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:30.886471 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1377/2780] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:46:30 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:30.890429 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa5e46-16e6-4258-9375-d67fabe7cccc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.901517 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa5e46-16e6-4258-9375-d67fabe7cccc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.955784 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730473-a302-4d68-87e4-fd723524ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:30.956300 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730473-a302-4d68-87e4-fd723524ad76 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:30.956577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730473-a302-4d68-87e4-fd723524ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:30.956845 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730473-a302-4d68-87e4-fd723524ad76 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:30.957511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d730473-a302-4d68-87e4-fd723524ad76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:30.959424 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d730473-a302-4d68-87e4-fd723524ad76 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:30.980737 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1404/2781] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:46:30 2022] POST /identity/v3/users => generated 306 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:30.984705 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5eb0a-fa7e-4585-916e-8acf6e613e66 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:30.995121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5eb0a-fa7e-4585-916e-8acf6e613e66 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:30.999779 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206948af-5242-4236-9812-d28279cd5e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:31.000381 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206948af-5242-4236-9812-d28279cd5e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:31.000678 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206948af-5242-4236-9812-d28279cd5e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:31.000954 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206948af-5242-4236-9812-d28279cd5e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:31.001602 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206948af-5242-4236-9812-d28279cd5e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:31.003176 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206948af-5242-4236-9812-d28279cd5e23 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:31.015707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1378/2782] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:31.019958 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730473-a302-4d68-87e4-fd723524ad76 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:31.030997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730473-a302-4d68-87e4-fd723524ad76 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:31.035692 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956153a-f19c-44ba-8215-b8e61f4b5c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:31.036342 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956153a-f19c-44ba-8215-b8e61f4b5c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:31.036620 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956153a-f19c-44ba-8215-b8e61f4b5c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:31.036892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956153a-f19c-44ba-8215-b8e61f4b5c42 admin admin] PATH_INFO: `/v3/projects/ac4a5e02818d49bc9f483e0493347339/users/86b13547fb204d8192bd56ab8ee90981/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:31.051308 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956153a-f19c-44ba-8215-b8e61f4b5c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4a5e02818d49bc9f483e0493347339, user_id=86b13547fb204d8192bd56ab8ee90981, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:31.054003 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956153a-f19c-44ba-8215-b8e61f4b5c42 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:31.062877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:31.067756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1405/2783] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:46:31 2022] PUT /identity/v3/projects/ac4a5e02818d49bc9f483e0493347339/users/86b13547fb204d8192bd56ab8ee90981/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:31.072370 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206948af-5242-4236-9812-d28279cd5e23 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:31.082590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206948af-5242-4236-9812-d28279cd5e23 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:31.138347 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c66467-abdd-43c6-bb52-a04f8624f169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:31.138989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c66467-abdd-43c6-bb52-a04f8624f169 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:31.139266 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c66467-abdd-43c6-bb52-a04f8624f169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:31.139533 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c66467-abdd-43c6-bb52-a04f8624f169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:31.140165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c66467-abdd-43c6-bb52-a04f8624f169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:31.141782 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c66467-abdd-43c6-bb52-a04f8624f169 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:31.152446 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1379/2784] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:31.156740 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956153a-f19c-44ba-8215-b8e61f4b5c42 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:31.169079 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956153a-f19c-44ba-8215-b8e61f4b5c42 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:31.174178 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:31.175034 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:31.175430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:31.175704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] PATH_INFO: `/v3/projects/ac4a5e02818d49bc9f483e0493347339/users/86b13547fb204d8192bd56ab8ee90981/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:31.178160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4a5e02818d49bc9f483e0493347339, user_id=86b13547fb204d8192bd56ab8ee90981, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:31.181589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:31.192884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1406/2785] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:46:31 2022] PUT /identity/v3/projects/ac4a5e02818d49bc9f483e0493347339/users/86b13547fb204d8192bd56ab8ee90981/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:31.203440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a8443-2f67-4481-b384-a119b0383e30 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:31.203720 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a8443-2f67-4481-b384-a119b0383e30 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:31.203994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a8443-2f67-4481-b384-a119b0383e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:31.234425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b11a8443-2f67-4481-b384-a119b0383e30 None None] MFA Rules not processed for user `86b13547fb204d8192bd56ab8ee90981`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:31.235558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b11a8443-2f67-4481-b384-a119b0383e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:31.303973 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1380/2786] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:31.315139 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:31.316890 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:31.399781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64019b4b-d318-4bb4-8324-e13e6852966c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:31.403446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10affe-25dd-46df-995e-6d849b5fbd53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:31.404041 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10affe-25dd-46df-995e-6d849b5fbd53 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:31.404318 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10affe-25dd-46df-995e-6d849b5fbd53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:31.404589 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10affe-25dd-46df-995e-6d849b5fbd53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:31.414313 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10affe-25dd-46df-995e-6d849b5fbd53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:31.415765 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10affe-25dd-46df-995e-6d849b5fbd53 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:31.425195 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1407/2787] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:31.501434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a8443-2f67-4481-b384-a119b0383e30 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:31.511817 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a8443-2f67-4481-b384-a119b0383e30 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:31.515582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76101a2f-93fd-48b7-85e9-6846049184ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:31.516183 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76101a2f-93fd-48b7-85e9-6846049184ec service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:31.516466 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76101a2f-93fd-48b7-85e9-6846049184ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:31.516765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76101a2f-93fd-48b7-85e9-6846049184ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:31.573412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76101a2f-93fd-48b7-85e9-6846049184ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:31.575125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76101a2f-93fd-48b7-85e9-6846049184ec service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:31.586634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1381/2788] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:31 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:36.757332 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10affe-25dd-46df-995e-6d849b5fbd53 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:36.819165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10affe-25dd-46df-995e-6d849b5fbd53 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:36.824255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:36.824867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:36.825142 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:36.825456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] PATH_INFO: `/v3/users/bd861479c1f5439bb4748cd7a4545e9e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:36.826558 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd861479c1f5439bb4748cd7a4545e9e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:36.828462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:36.879622 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd861479c1f5439bb4748cd7a4545e9e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:36.886648 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd861479c1f5439bb4748cd7a4545e9e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:36.887303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd861479c1f5439bb4748cd7a4545e9e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:36.899228 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd861479c1f5439bb4748cd7a4545e9e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:36.899850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd861479c1f5439bb4748cd7a4545e9e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:36.901005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:36.914250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1408/2789] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:46:36 2022] DELETE /identity/v3/users/bd861479c1f5439bb4748cd7a4545e9e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:36.919252 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809758-74ca-4ea5-ac62-465b4bc9a44d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:36.919725 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809758-74ca-4ea5-ac62-465b4bc9a44d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:36.919839 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8809758-74ca-4ea5-ac62-465b4bc9a44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:36.933295 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:36.941145 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b8809758-74ca-4ea5-ac62-465b4bc9a44d None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:36.942384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b8809758-74ca-4ea5-ac62-465b4bc9a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:37.001819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1382/2790] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:37.011505 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:37.012877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:37.069613 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506f089-1bfa-4280-a300-06493138c6ac admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:37.082258 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5b80e-c323-4fd9-9bc4-afc4b9152a37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:37.082875 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5b80e-c323-4fd9-9bc4-afc4b9152a37 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:37.083149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5b80e-c323-4fd9-9bc4-afc4b9152a37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:37.083418 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5b80e-c323-4fd9-9bc4-afc4b9152a37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:37.097149 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5b80e-c323-4fd9-9bc4-afc4b9152a37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:37.098626 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5b80e-c323-4fd9-9bc4-afc4b9152a37 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:37.109022 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1409/2791] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:37.401804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8809758-74ca-4ea5-ac62-465b4bc9a44d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:37.403322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b8809758-74ca-4ea5-ac62-465b4bc9a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:37.417761 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8809758-74ca-4ea5-ac62-465b4bc9a44d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:37.424517 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:37.425318 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:37.426157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:37.426157 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] PATH_INFO: `/v3/projects/35542546f3d74527a69742301c2085e3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:37.427366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35542546f3d74527a69742301c2085e3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:37.429578 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:37.488692 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35542546f3d74527a69742301c2085e3', 'request_id': 'req-6eab9317-fa7a-449c-a725-8bb44910d5c5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.495060 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35542546f3d74527a69742301c2085e3', 'request_id': 'req-6eab9317-fa7a-449c-a725-8bb44910d5c5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.502260 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35542546f3d74527a69742301c2085e3', 'request_id': 'req-6eab9317-fa7a-449c-a725-8bb44910d5c5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.515852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] The token cache is being invalidate because project 35542546f3d74527a69742301c2085e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:46:37.516218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.516928 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.518906 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1383/2792] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:46:37 2022] DELETE /identity/v3/projects/35542546f3d74527a69742301c2085e3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:37.522443 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5b80e-c323-4fd9-9bc4-afc4b9152a37 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:37.546364 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5b80e-c323-4fd9-9bc4-afc4b9152a37 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:37.617001 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:37.617706 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:37.617998 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:37.618274 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] PATH_INFO: `/v3/users/86b13547fb204d8192bd56ab8ee90981` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:37.619416 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b13547fb204d8192bd56ab8ee90981)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:37.621413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:37.670586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b13547fb204d8192bd56ab8ee90981'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.677862 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b13547fb204d8192bd56ab8ee90981'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.678532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b13547fb204d8192bd56ab8ee90981'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.692844 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b13547fb204d8192bd56ab8ee90981'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.693542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b8db25f3-14bd-4a2c-b4ff-cdb77571266c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b13547fb204d8192bd56ab8ee90981'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:37.706512 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1410/2793] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:46:37 2022] DELETE /identity/v3/users/86b13547fb204d8192bd56ab8ee90981 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:37.996019 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:37.997554 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:38.013111 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9317-fa7a-449c-a725-8bb44910d5c5 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:38.083440 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:38.085723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.086000 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.086265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] PATH_INFO: `/v3/projects/ac4a5e02818d49bc9f483e0493347339` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.088073 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4a5e02818d49bc9f483e0493347339)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:38.090021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:38.133265 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:38.148510 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4a5e02818d49bc9f483e0493347339', 'request_id': 'req-ac79739e-c0ff-4dd1-b033-e671a90dcacc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:38.154624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4a5e02818d49bc9f483e0493347339', 'request_id': 'req-ac79739e-c0ff-4dd1-b033-e671a90dcacc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:38.162090 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4a5e02818d49bc9f483e0493347339', 'request_id': 'req-ac79739e-c0ff-4dd1-b033-e671a90dcacc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:38.177355 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] The token cache is being invalidate because project ac4a5e02818d49bc9f483e0493347339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:46:38.177732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:38.178435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ac79739e-c0ff-4dd1-b033-e671a90dcacc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:38.180869 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1384/2794] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:46:37 2022] DELETE /identity/v3/projects/ac4a5e02818d49bc9f483e0493347339 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:38.185725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ab125-4d98-46bc-a835-c5a8d32857ed None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.186602 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ab125-4d98-46bc-a835-c5a8d32857ed None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.186901 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ab125-4d98-46bc-a835-c5a8d32857ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.218249 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c26ab125-4d98-46bc-a835-c5a8d32857ed None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:38.219942 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c26ab125-4d98-46bc-a835-c5a8d32857ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:38.226520 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:38.304460 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1411/2795] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:38.312353 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb855b6-1246-4159-a663-2366f65ac6f4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.312631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb855b6-1246-4159-a663-2366f65ac6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.312895 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb855b6-1246-4159-a663-2366f65ac6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.338403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5eb855b6-1246-4159-a663-2366f65ac6f4 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:38.339579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5eb855b6-1246-4159-a663-2366f65ac6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:38.349085 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1385/2796] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:38.354050 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ab125-4d98-46bc-a835-c5a8d32857ed None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:38.364310 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ab125-4d98-46bc-a835-c5a8d32857ed None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:38.377783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:38.378430 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.378705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.378968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.379612 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:46:38.379907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:38.381565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:38.398856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1412/2797] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:46:38.404008 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb855b6-1246-4159-a663-2366f65ac6f4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:38.418411 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb855b6-1246-4159-a663-2366f65ac6f4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:38.424412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fdf99-afb9-45c2-8d6a-460f8926b1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:38.425320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fdf99-afb9-45c2-8d6a-460f8926b1ac admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.425478 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fdf99-afb9-45c2-8d6a-460f8926b1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.425772 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fdf99-afb9-45c2-8d6a-460f8926b1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.426405 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fdf99-afb9-45c2-8d6a-460f8926b1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:38.428328 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fdf99-afb9-45c2-8d6a-460f8926b1ac admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:38.485317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1386/2798] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:46:38 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:38.489713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:38.502720 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6485ce8-29ad-4be3-80e6-756918e79094 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:38.564586 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce9906-7877-48da-8dd8-a47bb365dd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:38.565649 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce9906-7877-48da-8dd8-a47bb365dd04 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.565996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce9906-7877-48da-8dd8-a47bb365dd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.566334 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce9906-7877-48da-8dd8-a47bb365dd04 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.567038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce9906-7877-48da-8dd8-a47bb365dd04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:38.569087 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce9906-7877-48da-8dd8-a47bb365dd04 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:38.604411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1413/2799] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:46:38 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:38.605659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fdf99-afb9-45c2-8d6a-460f8926b1ac admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:38.617567 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fdf99-afb9-45c2-8d6a-460f8926b1ac admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:38.629453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5bda7-efd2-4c89-9dfe-cf0780487912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:38.630084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5bda7-efd2-4c89-9dfe-cf0780487912 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.630358 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5bda7-efd2-4c89-9dfe-cf0780487912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.630622 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5bda7-efd2-4c89-9dfe-cf0780487912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.631268 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5bda7-efd2-4c89-9dfe-cf0780487912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:38.632853 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5bda7-efd2-4c89-9dfe-cf0780487912 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:38.652087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1387/2800] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:38.653516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce9906-7877-48da-8dd8-a47bb365dd04 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:38.673975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce9906-7877-48da-8dd8-a47bb365dd04 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:38.685492 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fc2b7-8238-48b2-a7e9-b4ba588fd59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:38.686107 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fc2b7-8238-48b2-a7e9-b4ba588fd59e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.686384 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fc2b7-8238-48b2-a7e9-b4ba588fd59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.686655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fc2b7-8238-48b2-a7e9-b4ba588fd59e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.687309 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086fc2b7-8238-48b2-a7e9-b4ba588fd59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:38.688914 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086fc2b7-8238-48b2-a7e9-b4ba588fd59e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:38.702109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1414/2801] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:38.706659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5bda7-efd2-4c89-9dfe-cf0780487912 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:38.720228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5bda7-efd2-4c89-9dfe-cf0780487912 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:38.726348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60422c55-a79f-4b8f-b0cc-6f607a25b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:38.727136 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60422c55-a79f-4b8f-b0cc-6f607a25b9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.727448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60422c55-a79f-4b8f-b0cc-6f607a25b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.727792 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60422c55-a79f-4b8f-b0cc-6f607a25b9fd admin admin] PATH_INFO: `/v3/projects/114680cbb1704a4b8f6e445325a8a32b/users/8951ad6f443d4ac0964e1b747c3f02e1/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.748679 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60422c55-a79f-4b8f-b0cc-6f607a25b9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=114680cbb1704a4b8f6e445325a8a32b, user_id=8951ad6f443d4ac0964e1b747c3f02e1, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:38.752568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60422c55-a79f-4b8f-b0cc-6f607a25b9fd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:38.767220 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1388/2802] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:46:38 2022] PUT /identity/v3/projects/114680cbb1704a4b8f6e445325a8a32b/users/8951ad6f443d4ac0964e1b747c3f02e1/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:38.782510 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ec8da-4886-48b8-82ca-aab4d9c0d08f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.782952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ec8da-4886-48b8-82ca-aab4d9c0d08f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.783299 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ec8da-4886-48b8-82ca-aab4d9c0d08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.819767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-627ec8da-4886-48b8-82ca-aab4d9c0d08f None None] MFA Rules not processed for user `8951ad6f443d4ac0964e1b747c3f02e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:38.821222 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-627ec8da-4886-48b8-82ca-aab4d9c0d08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:38.903322 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1415/2803] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:38.908439 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60422c55-a79f-4b8f-b0cc-6f607a25b9fd admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:38.979299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60422c55-a79f-4b8f-b0cc-6f607a25b9fd admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:38.984307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3814ce-b568-4ca9-a9e5-73f401c450e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:38.984952 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3814ce-b568-4ca9-a9e5-73f401c450e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:38.985261 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3814ce-b568-4ca9-a9e5-73f401c450e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:38.985568 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3814ce-b568-4ca9-a9e5-73f401c450e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:38.986301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3814ce-b568-4ca9-a9e5-73f401c450e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:38.988590 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3814ce-b568-4ca9-a9e5-73f401c450e1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:39.016528 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1389/2804] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:46:38 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:39.020635 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ec8da-4886-48b8-82ca-aab4d9c0d08f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:39.032396 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ec8da-4886-48b8-82ca-aab4d9c0d08f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:39.036278 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:39.094411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d186b-0a89-49fb-af30-42b4dd9aef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:39.095094 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d186b-0a89-49fb-af30-42b4dd9aef88 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:39.095391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d186b-0a89-49fb-af30-42b4dd9aef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:39.095689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d186b-0a89-49fb-af30-42b4dd9aef88 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:39.096363 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d186b-0a89-49fb-af30-42b4dd9aef88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:39.098438 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d186b-0a89-49fb-af30-42b4dd9aef88 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:39.121557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1416/2805] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:46:39 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:46:39.125614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3814ce-b568-4ca9-a9e5-73f401c450e1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:39.136256 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3814ce-b568-4ca9-a9e5-73f401c450e1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:39.141702 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e98a7-bb2e-4d49-9115-b5dbbbd8d5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:39.142474 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e98a7-bb2e-4d49-9115-b5dbbbd8d5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:39.144325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e98a7-bb2e-4d49-9115-b5dbbbd8d5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:39.144325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e98a7-bb2e-4d49-9115-b5dbbbd8d5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:39.144325 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e98a7-bb2e-4d49-9115-b5dbbbd8d5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:39.145743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e98a7-bb2e-4d49-9115-b5dbbbd8d5c1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:39.158731 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1390/2806] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:39.162838 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d186b-0a89-49fb-af30-42b4dd9aef88 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:39.174910 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d186b-0a89-49fb-af30-42b4dd9aef88 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:39.180002 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87ae05-2e66-4636-a0d7-532f2e16dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:39.180868 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87ae05-2e66-4636-a0d7-532f2e16dcd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:39.181148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87ae05-2e66-4636-a0d7-532f2e16dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:39.181461 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87ae05-2e66-4636-a0d7-532f2e16dcd8 admin admin] PATH_INFO: `/v3/projects/459b10854ea845769a68fdf85fe65c44/users/777b47d04b0f4a3da2353c90a3885841/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:39.199511 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87ae05-2e66-4636-a0d7-532f2e16dcd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459b10854ea845769a68fdf85fe65c44, user_id=777b47d04b0f4a3da2353c90a3885841, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:39.202539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87ae05-2e66-4636-a0d7-532f2e16dcd8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:39.219386 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1417/2807] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:46:39 2022] PUT /identity/v3/projects/459b10854ea845769a68fdf85fe65c44/users/777b47d04b0f4a3da2353c90a3885841/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:46:39.223483 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e98a7-bb2e-4d49-9115-b5dbbbd8d5c1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:39.234461 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e98a7-bb2e-4d49-9115-b5dbbbd8d5c1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:39.265221 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:39.302548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3907-c12a-4bc0-9107-a696f20dcc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:39.303342 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3907-c12a-4bc0-9107-a696f20dcc96 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:39.303653 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3907-c12a-4bc0-9107-a696f20dcc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:39.303926 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3907-c12a-4bc0-9107-a696f20dcc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:39.304573 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb3907-c12a-4bc0-9107-a696f20dcc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:39.306245 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb3907-c12a-4bc0-9107-a696f20dcc96 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:39.318693 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1391/2808] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:46:39.323411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87ae05-2e66-4636-a0d7-532f2e16dcd8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:39.334456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87ae05-2e66-4636-a0d7-532f2e16dcd8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:39.339616 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:39.340268 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:39.340539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:39.340806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] PATH_INFO: `/v3/projects/459b10854ea845769a68fdf85fe65c44/users/777b47d04b0f4a3da2353c90a3885841/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:39.342752 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] RBAC: Authorizing `identity:create_grant(project_id=459b10854ea845769a68fdf85fe65c44, user_id=777b47d04b0f4a3da2353c90a3885841, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:39.345408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:39.359610 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1418/2809] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:46:39 2022] PUT /identity/v3/projects/459b10854ea845769a68fdf85fe65c44/users/777b47d04b0f4a3da2353c90a3885841/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:46:39.372447 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71f2ee-8ccc-48d3-88b9-173b51170f1b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:39.372732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71f2ee-8ccc-48d3-88b9-173b51170f1b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:39.373007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71f2ee-8ccc-48d3-88b9-173b51170f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:39.407205 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ad71f2ee-8ccc-48d3-88b9-173b51170f1b None None] MFA Rules not processed for user `777b47d04b0f4a3da2353c90a3885841`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:39.408438 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ad71f2ee-8ccc-48d3-88b9-173b51170f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:39.492560 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1392/2810] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:39.505134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:39.507541 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:39.570919 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721ea4c-9d7e-44b9-9bde-068b7221808b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:39.575294 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990c355-cb39-4640-b7cb-97bd86502a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:39.575897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990c355-cb39-4640-b7cb-97bd86502a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:39.576171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990c355-cb39-4640-b7cb-97bd86502a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:39.576437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990c355-cb39-4640-b7cb-97bd86502a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:39.585883 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990c355-cb39-4640-b7cb-97bd86502a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:39.587346 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990c355-cb39-4640-b7cb-97bd86502a2b service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:39.598864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1419/2811] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:39.713895 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71f2ee-8ccc-48d3-88b9-173b51170f1b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:39.727799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71f2ee-8ccc-48d3-88b9-173b51170f1b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:39.731495 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9c908-f96f-471d-8c21-7099c9754102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:39.732106 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9c908-f96f-471d-8c21-7099c9754102 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:39.732384 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9c908-f96f-471d-8c21-7099c9754102 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:39.732650 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9c908-f96f-471d-8c21-7099c9754102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:39.797701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9c908-f96f-471d-8c21-7099c9754102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:39.799158 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9c908-f96f-471d-8c21-7099c9754102 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:39.809872 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1393/2812] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:42.479210 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990c355-cb39-4640-b7cb-97bd86502a2b service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:42.541402 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990c355-cb39-4640-b7cb-97bd86502a2b service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:42.546073 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:42.546777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:42.547116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:42.547447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] PATH_INFO: `/v3/users/8951ad6f443d4ac0964e1b747c3f02e1` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:42.548675 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8951ad6f443d4ac0964e1b747c3f02e1)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:42.550661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:42.553183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:42.597916 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8951ad6f443d4ac0964e1b747c3f02e1'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:42.605733 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8951ad6f443d4ac0964e1b747c3f02e1'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:42.606355 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8951ad6f443d4ac0964e1b747c3f02e1'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:42.618304 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8951ad6f443d4ac0964e1b747c3f02e1'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:42.618938 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8951ad6f443d4ac0964e1b747c3f02e1'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:42.630783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1420/2813] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:46:42 2022] DELETE /identity/v3/users/8951ad6f443d4ac0964e1b747c3f02e1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:42.635419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c9449-a8fd-4321-905b-0ef52e070a80 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:42.635884 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c9449-a8fd-4321-905b-0ef52e070a80 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:42.635997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c9449-a8fd-4321-905b-0ef52e070a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:42.655226 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1c5c9449-a8fd-4321-905b-0ef52e070a80 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:42.656403 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1c5c9449-a8fd-4321-905b-0ef52e070a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:42.699433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:42.715415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1394/2814] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:42.725661 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:42.727038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:42.791642 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3841d6-334a-4d6d-95c0-4db8aa3f7407 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:42.795201 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936fac2-23d1-44c0-a38c-072bf446c497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:42.795804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936fac2-23d1-44c0-a38c-072bf446c497 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:42.796081 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936fac2-23d1-44c0-a38c-072bf446c497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:42.796351 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936fac2-23d1-44c0-a38c-072bf446c497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:42.805452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936fac2-23d1-44c0-a38c-072bf446c497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:42.806878 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936fac2-23d1-44c0-a38c-072bf446c497 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:42.816253 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1421/2815] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:46:43.107362 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c9449-a8fd-4321-905b-0ef52e070a80 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:43.109156 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1c5c9449-a8fd-4321-905b-0ef52e070a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:43.126154 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c9449-a8fd-4321-905b-0ef52e070a80 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:43.131768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:43.132246 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:43.132707 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:43.132882 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] PATH_INFO: `/v3/projects/114680cbb1704a4b8f6e445325a8a32b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:43.134669 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=114680cbb1704a4b8f6e445325a8a32b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:43.136668 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:43.183651 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '114680cbb1704a4b8f6e445325a8a32b', 'request_id': 'req-15109163-30e0-495e-9f42-2309cbfbb1e8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.189152 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '114680cbb1704a4b8f6e445325a8a32b', 'request_id': 'req-15109163-30e0-495e-9f42-2309cbfbb1e8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.195382 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '114680cbb1704a4b8f6e445325a8a32b', 'request_id': 'req-15109163-30e0-495e-9f42-2309cbfbb1e8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.208794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] The token cache is being invalidate because project 114680cbb1704a4b8f6e445325a8a32b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:46:43.209080 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.209743 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.211225 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1395/2816] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:46:43 2022] DELETE /identity/v3/projects/114680cbb1704a4b8f6e445325a8a32b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:43.215867 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936fac2-23d1-44c0-a38c-072bf446c497 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:43.225491 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936fac2-23d1-44c0-a38c-072bf446c497 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:43.287196 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:43.287962 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:43.288248 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:43.288513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] PATH_INFO: `/v3/users/777b47d04b0f4a3da2353c90a3885841` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:43.289679 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=777b47d04b0f4a3da2353c90a3885841)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:43.291596 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:43.324564 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:43.344801 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '777b47d04b0f4a3da2353c90a3885841'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.352565 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '777b47d04b0f4a3da2353c90a3885841'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.353165 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '777b47d04b0f4a3da2353c90a3885841'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.365444 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '777b47d04b0f4a3da2353c90a3885841'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.366105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-126fd333-6838-4af0-9a48-a9983dca6698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '777b47d04b0f4a3da2353c90a3885841'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.378819 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1422/2817] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:46:43 2022] DELETE /identity/v3/users/777b47d04b0f4a3da2353c90a3885841 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:43.707024 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:43.708920 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:43.725994 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15109163-30e0-495e-9f42-2309cbfbb1e8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:43.814277 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:43.814944 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:43.815203 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:43.815468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] PATH_INFO: `/v3/projects/459b10854ea845769a68fdf85fe65c44` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:43.816607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] RBAC: Authorizing `identity:delete_project(project_id=459b10854ea845769a68fdf85fe65c44)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:43.818562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:43.870109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459b10854ea845769a68fdf85fe65c44', 'request_id': 'req-6288a6f0-4119-4557-be3d-a7f9cf7c630d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.875961 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459b10854ea845769a68fdf85fe65c44', 'request_id': 'req-6288a6f0-4119-4557-be3d-a7f9cf7c630d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.882710 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459b10854ea845769a68fdf85fe65c44', 'request_id': 'req-6288a6f0-4119-4557-be3d-a7f9cf7c630d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.899005 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] The token cache is being invalidate because project 459b10854ea845769a68fdf85fe65c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:46:43.899970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.900582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6288a6f0-4119-4557-be3d-a7f9cf7c630d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:46:43.902488 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1396/2818] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:46:43 2022] DELETE /identity/v3/projects/459b10854ea845769a68fdf85fe65c44 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:43.907421 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c4c38-a01b-4ce2-884a-52c47bc52798 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:43.907966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c4c38-a01b-4ce2-884a-52c47bc52798 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:43.908090 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c4c38-a01b-4ce2-884a-52c47bc52798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:43.927574 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e44c4c38-a01b-4ce2-884a-52c47bc52798 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:43.928913 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e44c4c38-a01b-4ce2-884a-52c47bc52798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:43.991725 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1423/2819] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:44.000721 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beccbb-a954-4499-ad53-7f3d53a5692b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.000998 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beccbb-a954-4499-ad53-7f3d53a5692b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.001321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beccbb-a954-4499-ad53-7f3d53a5692b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.010931 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:44.022690 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c0beccbb-a954-4499-ad53-7f3d53a5692b None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:44.025857 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c0beccbb-a954-4499-ad53-7f3d53a5692b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:44.034631 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1397/2820] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:44.039049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c4c38-a01b-4ce2-884a-52c47bc52798 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:44.049109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c4c38-a01b-4ce2-884a-52c47bc52798 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:44.053695 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:44.054378 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.054656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.054993 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.056192 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:46:44.056549 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:44.059232 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:44.072425 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1424/2821] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:46:44.076923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0beccbb-a954-4499-ad53-7f3d53a5692b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:44.086327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0beccbb-a954-4499-ad53-7f3d53a5692b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:44.090963 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa651bc3-17b7-4ad8-95d8-91da7c193450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:44.091555 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa651bc3-17b7-4ad8-95d8-91da7c193450 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.091827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa651bc3-17b7-4ad8-95d8-91da7c193450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.092087 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa651bc3-17b7-4ad8-95d8-91da7c193450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.092704 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa651bc3-17b7-4ad8-95d8-91da7c193450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:44.094605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa651bc3-17b7-4ad8-95d8-91da7c193450 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:44.126233 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1398/2822] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:46:44 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:46:44.131380 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:44.142473 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353106de-a72b-4798-b370-be6de06852bb admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:44.198043 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8857cc-4dbe-42b3-b60c-51385ab5f54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:44.198673 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8857cc-4dbe-42b3-b60c-51385ab5f54a admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.198951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8857cc-4dbe-42b3-b60c-51385ab5f54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.199223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8857cc-4dbe-42b3-b60c-51385ab5f54a admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.199841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8857cc-4dbe-42b3-b60c-51385ab5f54a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:44.201800 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8857cc-4dbe-42b3-b60c-51385ab5f54a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:44.222270 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1425/2823] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:46:44 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:46:44.226566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa651bc3-17b7-4ad8-95d8-91da7c193450 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:44.236635 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa651bc3-17b7-4ad8-95d8-91da7c193450 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:44.241518 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f961022-99eb-48d6-8e45-18e8fed8cad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:44.242218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f961022-99eb-48d6-8e45-18e8fed8cad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.242385 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f961022-99eb-48d6-8e45-18e8fed8cad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.242703 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f961022-99eb-48d6-8e45-18e8fed8cad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.243892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f961022-99eb-48d6-8e45-18e8fed8cad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:44.245504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f961022-99eb-48d6-8e45-18e8fed8cad2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:44.255686 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1399/2824] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:44.260382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8857cc-4dbe-42b3-b60c-51385ab5f54a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:44.270354 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8857cc-4dbe-42b3-b60c-51385ab5f54a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:44.275118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a481f58-d086-4364-9e1d-d00a945f575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:44.275713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a481f58-d086-4364-9e1d-d00a945f575e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.276005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a481f58-d086-4364-9e1d-d00a945f575e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.276250 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a481f58-d086-4364-9e1d-d00a945f575e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.276885 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a481f58-d086-4364-9e1d-d00a945f575e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:44.278631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a481f58-d086-4364-9e1d-d00a945f575e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:44.289252 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1426/2825] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:46:44.293607 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f961022-99eb-48d6-8e45-18e8fed8cad2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:44.303516 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f961022-99eb-48d6-8e45-18e8fed8cad2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:44.308586 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:44.309255 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.309542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.309815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] PATH_INFO: `/v3/projects/4602b7775b0c4960a7a1a781644b0225/users/a4aa28893125426fa7aef9b83781f373/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.330187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4602b7775b0c4960a7a1a781644b0225, user_id=a4aa28893125426fa7aef9b83781f373, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:44.332844 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:44.346084 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1400/2826] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:46:44 2022] PUT /identity/v3/projects/4602b7775b0c4960a7a1a781644b0225/users/a4aa28893125426fa7aef9b83781f373/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:46:44.412411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ef079-549b-4b60-b481-60abea81c1df None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.412713 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ef079-549b-4b60-b481-60abea81c1df None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.412983 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ef079-549b-4b60-b481-60abea81c1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.434431 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:44.446415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-657ef079-549b-4b60-b481-60abea81c1df None None] MFA Rules not processed for user `a4aa28893125426fa7aef9b83781f373`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:46:44.448772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-657ef079-549b-4b60-b481-60abea81c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:44.516767 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1427/2827] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:46:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:46:44.527891 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:46:44.529611 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:46:44.587941 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265e8af-9ecd-4269-9516-03e34e1fe895 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:46:44.591549 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:46:44.592148 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:46:44.592420 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:46:44.592685 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:46:44.600654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:46:44.602187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:46:44.610769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1401/2828] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:46:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:46:54.514207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=28 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:46:54.607127 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:14.144272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7851abe-afd5-4074-871f-8616055db13f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:14.144964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7851abe-afd5-4074-871f-8616055db13f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:14.144964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7851abe-afd5-4074-871f-8616055db13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:14.165011 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f7851abe-afd5-4074-871f-8616055db13f None None] MFA Rules not processed for user `0968c56b8be242918903711961b6d5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:14.166303 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f7851abe-afd5-4074-871f-8616055db13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:14.252025 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1428/2829] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:47:14.371251 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:14.372802 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:14.474706 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159396c-e86a-458f-aa34-b198578b89e3 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:14.483242 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:14.484341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:14.484341 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:14.485408 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] PATH_INFO: `/v3/users/d7bd571bcb71452299829b7dcda8f944` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:14.486871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7bd571bcb71452299829b7dcda8f944)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:14.490634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:14.582605 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7bd571bcb71452299829b7dcda8f944'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:14.604755 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7bd571bcb71452299829b7dcda8f944'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:14.605889 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7bd571bcb71452299829b7dcda8f944'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:14.623356 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7bd571bcb71452299829b7dcda8f944'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:14.639798 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7bd571bcb71452299829b7dcda8f944'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:14.647320 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1402/2830] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:47:14 2022] DELETE /identity/v3/users/d7bd571bcb71452299829b7dcda8f944 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:14.652889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60f7f7-5d59-49be-ad99-c3a52b5e4955 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:14.653280 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60f7f7-5d59-49be-ad99-c3a52b5e4955 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:14.653555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60f7f7-5d59-49be-ad99-c3a52b5e4955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:14.679113 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4b60f7f7-5d59-49be-ad99-c3a52b5e4955 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:14.680555 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4b60f7f7-5d59-49be-ad99-c3a52b5e4955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:14.769130 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1429/2831] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:14.809317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:14.811204 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:14.902425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c2d69-58d0-4252-bd35-7007b98fc229 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:14.907713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184befc3-3cc6-449a-a5a1-1a2938abef60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:14.908924 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184befc3-3cc6-449a-a5a1-1a2938abef60 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:14.909636 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184befc3-3cc6-449a-a5a1-1a2938abef60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:14.910301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184befc3-3cc6-449a-a5a1-1a2938abef60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:14.927634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184befc3-3cc6-449a-a5a1-1a2938abef60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:14.930228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184befc3-3cc6-449a-a5a1-1a2938abef60 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:14.949094 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1403/2832] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:47:15.734534 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60f7f7-5d59-49be-ad99-c3a52b5e4955 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:15.738118 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4b60f7f7-5d59-49be-ad99-c3a52b5e4955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:15.753677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60f7f7-5d59-49be-ad99-c3a52b5e4955 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:15.759507 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:15.760160 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:15.760634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:15.762055 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] PATH_INFO: `/v3/projects/b48a8b2af6ad4f59a6b5ee3fcddd66a8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:15.764053 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48a8b2af6ad4f59a6b5ee3fcddd66a8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:15.769454 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:15.847162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48a8b2af6ad4f59a6b5ee3fcddd66a8', 'request_id': 'req-7485a6a9-9168-401b-a8fe-22d4528bfc8d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:15.856089 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48a8b2af6ad4f59a6b5ee3fcddd66a8', 'request_id': 'req-7485a6a9-9168-401b-a8fe-22d4528bfc8d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:15.867542 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48a8b2af6ad4f59a6b5ee3fcddd66a8', 'request_id': 'req-7485a6a9-9168-401b-a8fe-22d4528bfc8d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:15.884412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] The token cache is being invalidate because project b48a8b2af6ad4f59a6b5ee3fcddd66a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:47:15.884933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:15.885724 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7485a6a9-9168-401b-a8fe-22d4528bfc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:15.887379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1430/2833] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:47:15 2022] DELETE /identity/v3/projects/b48a8b2af6ad4f59a6b5ee3fcddd66a8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:47:15.892799 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae942b-dcc9-421b-a014-c084ca7f08e7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:15.893390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae942b-dcc9-421b-a014-c084ca7f08e7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:15.893597 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae942b-dcc9-421b-a014-c084ca7f08e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:15.922215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7fae942b-dcc9-421b-a014-c084ca7f08e7 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:15.923425 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7fae942b-dcc9-421b-a014-c084ca7f08e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:15.943460 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:15.991575 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1404/2834] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:47:16.004795 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de423-e4b1-464b-8b3e-69c8aa972423 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.005114 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de423-e4b1-464b-8b3e-69c8aa972423 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.005368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de423-e4b1-464b-8b3e-69c8aa972423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.025811 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-941de423-e4b1-464b-8b3e-69c8aa972423 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:16.027046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-941de423-e4b1-464b-8b3e-69c8aa972423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:16.035218 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1431/2835] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:16.040677 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae942b-dcc9-421b-a014-c084ca7f08e7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:16.056390 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae942b-dcc9-421b-a014-c084ca7f08e7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:16.064178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:16.064178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.064178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.064178 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.066285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:47:16.066285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:16.066769 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:16.083193 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1405/2836] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:47:16.087689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941de423-e4b1-464b-8b3e-69c8aa972423 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:16.100773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941de423-e4b1-464b-8b3e-69c8aa972423 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:16.108783 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29974691-f8fd-4ca3-a119-b397452ea314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:16.109808 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974691-f8fd-4ca3-a119-b397452ea314 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.110290 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974691-f8fd-4ca3-a119-b397452ea314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.110763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974691-f8fd-4ca3-a119-b397452ea314 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.111905 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29974691-f8fd-4ca3-a119-b397452ea314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:16.115333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29974691-f8fd-4ca3-a119-b397452ea314 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:16.157982 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:16.183468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:16.184977 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1432/2837] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:47:16 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:47:16.198871 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aed682-ab42-483d-80f8-013b8fed4eb1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:16.329021 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3235f-f056-41a8-9d0d-abb698996d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:16.330229 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3235f-f056-41a8-9d0d-abb698996d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.330493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3235f-f056-41a8-9d0d-abb698996d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.330763 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3235f-f056-41a8-9d0d-abb698996d3d admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.331391 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3235f-f056-41a8-9d0d-abb698996d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:16.333334 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3235f-f056-41a8-9d0d-abb698996d3d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:16.374335 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1406/2838] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:47:16 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:47:16.375415 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29974691-f8fd-4ca3-a119-b397452ea314 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:16.386298 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29974691-f8fd-4ca3-a119-b397452ea314 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:16.394036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fe12e-7aa0-4bf7-864d-8f092414f49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:16.394895 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe12e-7aa0-4bf7-864d-8f092414f49b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.395144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe12e-7aa0-4bf7-864d-8f092414f49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.395516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe12e-7aa0-4bf7-864d-8f092414f49b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.396464 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fe12e-7aa0-4bf7-864d-8f092414f49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:16.398979 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fe12e-7aa0-4bf7-864d-8f092414f49b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:16.423667 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1433/2839] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:47:16.428589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3235f-f056-41a8-9d0d-abb698996d3d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:16.450301 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3235f-f056-41a8-9d0d-abb698996d3d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:16.459118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2287f85-eeec-4ed0-a4c7-a166253fc27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:16.460150 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2287f85-eeec-4ed0-a4c7-a166253fc27f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.461097 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2287f85-eeec-4ed0-a4c7-a166253fc27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.461624 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2287f85-eeec-4ed0-a4c7-a166253fc27f admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.462629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2287f85-eeec-4ed0-a4c7-a166253fc27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:16.465133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2287f85-eeec-4ed0-a4c7-a166253fc27f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:16.485822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1407/2840] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:47:16.490177 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fe12e-7aa0-4bf7-864d-8f092414f49b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:16.506516 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fe12e-7aa0-4bf7-864d-8f092414f49b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:16.514843 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:16.515964 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.516437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.516912 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] PATH_INFO: `/v3/projects/6417b684d63549d8ba6796f0d2abf826/users/5f37c8a833884ae98fe12de6e5f37307/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.546333 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6417b684d63549d8ba6796f0d2abf826, user_id=5f37c8a833884ae98fe12de6e5f37307, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:16.551243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:16.587751 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55423c49-ebdc-465f-bcac-dffe5ec724f1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.588105 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55423c49-ebdc-465f-bcac-dffe5ec724f1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.588492 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55423c49-ebdc-465f-bcac-dffe5ec724f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.591659 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1434/2841] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:47:16 2022] PUT /identity/v3/projects/6417b684d63549d8ba6796f0d2abf826/users/5f37c8a833884ae98fe12de6e5f37307/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:16.646566 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-55423c49-ebdc-465f-bcac-dffe5ec724f1 None None] MFA Rules not processed for user `5f37c8a833884ae98fe12de6e5f37307`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:16.649313 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-55423c49-ebdc-465f-bcac-dffe5ec724f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:16.798652 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1408/2842] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:47:16.813169 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:16.815028 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:16.953078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc2a97-41bd-4b1b-b7da-a912f4b3c5fb admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:16.957726 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c22c1d-ae9d-442e-a4f4-9dffc1042212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:16.958487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c22c1d-ae9d-442e-a4f4-9dffc1042212 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:16.958763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c22c1d-ae9d-442e-a4f4-9dffc1042212 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:16.959361 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c22c1d-ae9d-442e-a4f4-9dffc1042212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:16.974433 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c22c1d-ae9d-442e-a4f4-9dffc1042212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:16.975923 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c22c1d-ae9d-442e-a4f4-9dffc1042212 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:16.986704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1435/2843] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:47:17.016307 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55423c49-ebdc-465f-bcac-dffe5ec724f1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:17.037696 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:17.100784 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55423c49-ebdc-465f-bcac-dffe5ec724f1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:17.106058 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:17.107121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:17.107493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:17.107768 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] PATH_INFO: `/v3/users/5f37c8a833884ae98fe12de6e5f37307` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:17.108923 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f37c8a833884ae98fe12de6e5f37307)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:17.110892 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:17.179790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f37c8a833884ae98fe12de6e5f37307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.189979 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f37c8a833884ae98fe12de6e5f37307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.190701 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f37c8a833884ae98fe12de6e5f37307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.201540 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f37c8a833884ae98fe12de6e5f37307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.218166 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f37c8a833884ae98fe12de6e5f37307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.224608 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1409/2844] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:47:17 2022] DELETE /identity/v3/users/5f37c8a833884ae98fe12de6e5f37307 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:17.231975 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf9bcb-3551-4402-929e-91ef03da5746 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:17.232462 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf9bcb-3551-4402-929e-91ef03da5746 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:17.232951 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf9bcb-3551-4402-929e-91ef03da5746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:17.268046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8ccf9bcb-3551-4402-929e-91ef03da5746 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:17.269395 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8ccf9bcb-3551-4402-929e-91ef03da5746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:17.366392 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1436/2845] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:17.375523 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:17.378841 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:17.449430 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f7a21-c77e-4bc9-a509-864280b2123d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:17.453435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e0f74-45e4-477f-ac5b-32bd0f1249d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:17.454137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e0f74-45e4-477f-ac5b-32bd0f1249d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:17.454527 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e0f74-45e4-477f-ac5b-32bd0f1249d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:17.455149 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e0f74-45e4-477f-ac5b-32bd0f1249d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:17.471728 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e0f74-45e4-477f-ac5b-32bd0f1249d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:17.474321 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e0f74-45e4-477f-ac5b-32bd0f1249d3 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:17.498654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1410/2846] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:47:17.523918 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf9bcb-3551-4402-929e-91ef03da5746 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:17.526301 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8ccf9bcb-3551-4402-929e-91ef03da5746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:17.544353 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf9bcb-3551-4402-929e-91ef03da5746 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:17.550504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:17.551247 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:17.551525 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:17.551791 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] PATH_INFO: `/v3/users/a4aa28893125426fa7aef9b83781f373` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:17.553035 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4aa28893125426fa7aef9b83781f373)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:17.555005 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:17.625655 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4aa28893125426fa7aef9b83781f373'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.636776 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4aa28893125426fa7aef9b83781f373'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.638101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4aa28893125426fa7aef9b83781f373'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.650065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:17.658014 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4aa28893125426fa7aef9b83781f373'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.658943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4aa28893125426fa7aef9b83781f373'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.685287 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1437/2847] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:47:17 2022] DELETE /identity/v3/users/a4aa28893125426fa7aef9b83781f373 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:17.689637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a651d6c-74eb-4974-b194-51f8cb0e65d9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:17.689637 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a651d6c-74eb-4974-b194-51f8cb0e65d9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:17.689980 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a651d6c-74eb-4974-b194-51f8cb0e65d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:17.714322 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5a651d6c-74eb-4974-b194-51f8cb0e65d9 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:17.715536 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5a651d6c-74eb-4974-b194-51f8cb0e65d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:17.765655 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:17.802811 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1411/2848] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:17.815377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:17.817065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:17.844888 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a651d6c-74eb-4974-b194-51f8cb0e65d9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:17.846804 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5a651d6c-74eb-4974-b194-51f8cb0e65d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:17.863342 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a651d6c-74eb-4974-b194-51f8cb0e65d9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:17.872083 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:17.873615 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:17.873741 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:17.874371 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] PATH_INFO: `/v3/projects/6417b684d63549d8ba6796f0d2abf826` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:17.876732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6417b684d63549d8ba6796f0d2abf826)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:17.879989 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:17.899889 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d23fb3-a0dc-42b0-8ada-c59fb9d79ad1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:17.906892 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64a707-0c90-44d5-b916-ef8f0b9e61b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:17.907909 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64a707-0c90-44d5-b916-ef8f0b9e61b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:17.908124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64a707-0c90-44d5-b916-ef8f0b9e61b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:17.908420 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64a707-0c90-44d5-b916-ef8f0b9e61b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:17.924607 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64a707-0c90-44d5-b916-ef8f0b9e61b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:17.926260 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64a707-0c90-44d5-b916-ef8f0b9e61b7 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:17.937116 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1438/2849] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:47:17.938901 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6417b684d63549d8ba6796f0d2abf826', 'request_id': 'req-73246a9d-892e-4300-b9e5-16e26826d775'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.945890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6417b684d63549d8ba6796f0d2abf826', 'request_id': 'req-73246a9d-892e-4300-b9e5-16e26826d775'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.953654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6417b684d63549d8ba6796f0d2abf826', 'request_id': 'req-73246a9d-892e-4300-b9e5-16e26826d775'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.972195 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] The token cache is being invalidate because project 6417b684d63549d8ba6796f0d2abf826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:47:17.972481 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.973401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-73246a9d-892e-4300-b9e5-16e26826d775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:17.974947 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1412/2850] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:47:17 2022] DELETE /identity/v3/projects/6417b684d63549d8ba6796f0d2abf826 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:17.987181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbbaca-a572-4057-beef-f1856827d9a5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:17.988379 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbbaca-a572-4057-beef-f1856827d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:17.988992 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbbaca-a572-4057-beef-f1856827d9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.015502 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8ecbbaca-a572-4057-beef-f1856827d9a5 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:18.017582 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8ecbbaca-a572-4057-beef-f1856827d9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:18.100206 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1439/2851] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:18.110808 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7556b1-f4d9-459b-985f-0ab596532a54 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.111448 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7556b1-f4d9-459b-985f-0ab596532a54 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.111660 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7556b1-f4d9-459b-985f-0ab596532a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.145942 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0c7556b1-f4d9-459b-985f-0ab596532a54 None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:18.147285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0c7556b1-f4d9-459b-985f-0ab596532a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:18.162581 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1413/2852] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:18.166341 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbbaca-a572-4057-beef-f1856827d9a5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:18.177033 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbbaca-a572-4057-beef-f1856827d9a5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:18.183426 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:18.184083 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.184366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.184633 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.185324 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:47:18.185619 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:18.187368 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:18.202407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1440/2853] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 10 13:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:47:18.206819 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7556b1-f4d9-459b-985f-0ab596532a54 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:18.219538 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7556b1-f4d9-459b-985f-0ab596532a54 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:18.229288 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:18.231102 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.231644 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.232218 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.234081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:18.237124 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:18.283417 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1414/2854] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:47:18 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:47:18.284873 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:18.310456 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754f7f0-3382-49f0-b5de-fa796ca95b41 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:18.369013 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:18.372239 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:18.417762 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79333d81-43c7-4f05-9d2a-34aa0696f8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:18.418756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79333d81-43c7-4f05-9d2a-34aa0696f8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.418756 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79333d81-43c7-4f05-9d2a-34aa0696f8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.419184 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79333d81-43c7-4f05-9d2a-34aa0696f8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.419907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79333d81-43c7-4f05-9d2a-34aa0696f8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:18.422588 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79333d81-43c7-4f05-9d2a-34aa0696f8e8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:18.442722 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:18.455483 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1441/2855] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:47:18 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:47:18.457447 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79333d81-43c7-4f05-9d2a-34aa0696f8e8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:18.471611 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79333d81-43c7-4f05-9d2a-34aa0696f8e8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:18.472628 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32813185-3f60-43ed-95fd-3d9832320ec3 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:18.477724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:18.478171 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65cf5-1e15-4d46-9fd9-631299760c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:18.478433 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.478640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.478912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] PATH_INFO: `/v3/projects/4602b7775b0c4960a7a1a781644b0225` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.479101 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65cf5-1e15-4d46-9fd9-631299760c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.479288 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65cf5-1e15-4d46-9fd9-631299760c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.479685 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65cf5-1e15-4d46-9fd9-631299760c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.480187 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4602b7775b0c4960a7a1a781644b0225)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:18.480408 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef65cf5-1e15-4d46-9fd9-631299760c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:18.482118 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:18.482307 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef65cf5-1e15-4d46-9fd9-631299760c03 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:18.494750 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1442/2856] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:47:18.499049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65cf5-1e15-4d46-9fd9-631299760c03 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:18.516583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65cf5-1e15-4d46-9fd9-631299760c03 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:18.525820 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279e2d8-b5b3-427d-856b-4b28daaa0b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:18.526435 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279e2d8-b5b3-427d-856b-4b28daaa0b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.526711 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279e2d8-b5b3-427d-856b-4b28daaa0b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.526995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279e2d8-b5b3-427d-856b-4b28daaa0b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.527609 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279e2d8-b5b3-427d-856b-4b28daaa0b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:18.530016 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279e2d8-b5b3-427d-856b-4b28daaa0b32 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:18.545539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1443/2857] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:47:18.546562 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279e2d8-b5b3-427d-856b-4b28daaa0b32 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:18.555125 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:18.558366 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279e2d8-b5b3-427d-856b-4b28daaa0b32 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:18.570249 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4602b7775b0c4960a7a1a781644b0225', 'request_id': 'req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:18.579709 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4602b7775b0c4960a7a1a781644b0225', 'request_id': 'req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:18.588997 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4602b7775b0c4960a7a1a781644b0225', 'request_id': 'req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:18.611402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] The token cache is being invalidate because project 4602b7775b0c4960a7a1a781644b0225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:47:18.612165 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:18.613634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5ddc409c-b81b-4d15-a2a8-fdeb02dcd735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:18.625480 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1415/2858] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:47:18 2022] DELETE /identity/v3/projects/4602b7775b0c4960a7a1a781644b0225 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:47:18.634411 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:18.635451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.635825 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.635997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] PATH_INFO: `/v3/projects/15d732726c004517ad7e771af3f3b805/users/0a7f0324eba3466aa985febfaf47b990/roles/49b24f3e5ebc4b08b538432eda6b4b5d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.655060 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d732726c004517ad7e771af3f3b805, user_id=0a7f0324eba3466aa985febfaf47b990, role_id=49b24f3e5ebc4b08b538432eda6b4b5d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:18.657758 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:18.675831 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1444/2859] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:47:18 2022] PUT /identity/v3/projects/15d732726c004517ad7e771af3f3b805/users/0a7f0324eba3466aa985febfaf47b990/roles/49b24f3e5ebc4b08b538432eda6b4b5d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:18.683829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8deaeb-d56d-4faa-a648-6b8d00dabd2a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.684366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8deaeb-d56d-4faa-a648-6b8d00dabd2a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.684733 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8deaeb-d56d-4faa-a648-6b8d00dabd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.709096 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fc8deaeb-d56d-4faa-a648-6b8d00dabd2a None None] MFA Rules not processed for user `1d95623897cc4c628d91416fe290bfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:18.710402 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fc8deaeb-d56d-4faa-a648-6b8d00dabd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:18.794053 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1416/2860] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:18.806097 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:18.808283 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:18.890689 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56f23a-33c0-4788-977f-35a9fbb9a211 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:18.901577 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:18.902770 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:18.903255 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:18.903732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:18.914877 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:18.916382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:18.926623 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1445/2861] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:47:25.803524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d891d-fd88-4b0e-b8f7-dd1c54081e71 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:25.803524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d891d-fd88-4b0e-b8f7-dd1c54081e71 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:25.803524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d891d-fd88-4b0e-b8f7-dd1c54081e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:25.858630 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-234d891d-fd88-4b0e-b8f7-dd1c54081e71 None None] MFA Rules not processed for user `0a7f0324eba3466aa985febfaf47b990`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:25.860077 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-234d891d-fd88-4b0e-b8f7-dd1c54081e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:25.969215 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1417/2862] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:25.973459 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:25.978124 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:26.003937 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3aad6-7838-417c-b1ce-8304cebb90a4 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:26.012162 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465761ad-d22d-4dad-9ba2-3764a11da04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:26.013190 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465761ad-d22d-4dad-9ba2-3764a11da04f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:26.013781 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465761ad-d22d-4dad-9ba2-3764a11da04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:26.014314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465761ad-d22d-4dad-9ba2-3764a11da04f admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:26.015108 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465761ad-d22d-4dad-9ba2-3764a11da04f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:26.017450 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465761ad-d22d-4dad-9ba2-3764a11da04f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:26.071357 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1446/2863] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 10 13:47:25 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:47:26.075544 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d891d-fd88-4b0e-b8f7-dd1c54081e71 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:26.087069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d891d-fd88-4b0e-b8f7-dd1c54081e71 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:26.170785 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:26.193852 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15332cc-b2c3-422f-a406-2ebed5d2a251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:26.195291 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15332cc-b2c3-422f-a406-2ebed5d2a251 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:26.195723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15332cc-b2c3-422f-a406-2ebed5d2a251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:26.196140 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15332cc-b2c3-422f-a406-2ebed5d2a251 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:26.197110 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15332cc-b2c3-422f-a406-2ebed5d2a251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:26.200299 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15332cc-b2c3-422f-a406-2ebed5d2a251 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:26.245308 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1418/2864] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 10 13:47:26 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:47:26.251013 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465761ad-d22d-4dad-9ba2-3764a11da04f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:26.284941 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465761ad-d22d-4dad-9ba2-3764a11da04f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:26.293295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98013d-2ab6-4718-9900-c1f1d599a30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:26.294109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98013d-2ab6-4718-9900-c1f1d599a30b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:26.294709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98013d-2ab6-4718-9900-c1f1d599a30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:26.295199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98013d-2ab6-4718-9900-c1f1d599a30b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:26.296330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98013d-2ab6-4718-9900-c1f1d599a30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:26.299150 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98013d-2ab6-4718-9900-c1f1d599a30b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:26.317159 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1447/2865] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:47:26.322175 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15332cc-b2c3-422f-a406-2ebed5d2a251 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:26.340742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15332cc-b2c3-422f-a406-2ebed5d2a251 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:26.347742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec0769-7bd2-4d20-9331-ff7334b2c6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:26.349293 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec0769-7bd2-4d20-9331-ff7334b2c6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:26.349491 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec0769-7bd2-4d20-9331-ff7334b2c6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:26.349801 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec0769-7bd2-4d20-9331-ff7334b2c6eb admin admin] PATH_INFO: `/v3/projects/8dfcadebf5bc4ef2b2469e269f5a743f/users/eb0927ae32a441c7acffa98f1901a800/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:26.390723 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec0769-7bd2-4d20-9331-ff7334b2c6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dfcadebf5bc4ef2b2469e269f5a743f, user_id=eb0927ae32a441c7acffa98f1901a800, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:26.397104 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec0769-7bd2-4d20-9331-ff7334b2c6eb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:26.427629 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1419/2866] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:47:26 2022] PUT /identity/v3/projects/8dfcadebf5bc4ef2b2469e269f5a743f/users/eb0927ae32a441c7acffa98f1901a800/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:47:26.432963 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98013d-2ab6-4718-9900-c1f1d599a30b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:26.460852 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98013d-2ab6-4718-9900-c1f1d599a30b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:26.486455 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:26.563109 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:26.563763 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:26.564046 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:26.564315 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:26.564959 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:26.566779 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:26.591102 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1448/2867] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 10 13:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:47:26.595823 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec0769-7bd2-4d20-9331-ff7334b2c6eb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:26.610269 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec0769-7bd2-4d20-9331-ff7334b2c6eb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:26.617274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c85ec-37af-4831-a2e5-a091b5aa1003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:26.618177 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c85ec-37af-4831-a2e5-a091b5aa1003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:26.618634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c85ec-37af-4831-a2e5-a091b5aa1003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:26.619069 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c85ec-37af-4831-a2e5-a091b5aa1003 admin admin] PATH_INFO: `/v3/projects/8dfcadebf5bc4ef2b2469e269f5a743f/users/eb0927ae32a441c7acffa98f1901a800/roles/f1cc6c5dc9634036b4adb0495566b562` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:26.621753 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c85ec-37af-4831-a2e5-a091b5aa1003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dfcadebf5bc4ef2b2469e269f5a743f, user_id=eb0927ae32a441c7acffa98f1901a800, role_id=f1cc6c5dc9634036b4adb0495566b562)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:26.625453 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c85ec-37af-4831-a2e5-a091b5aa1003 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:26.657038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1420/2868] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 10 13:47:26 2022] PUT /identity/v3/projects/8dfcadebf5bc4ef2b2469e269f5a743f/users/eb0927ae32a441c7acffa98f1901a800/roles/f1cc6c5dc9634036b4adb0495566b562 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:47:29.303860 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:29.306760 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:29.492841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e4a5e-5c92-4546-92ac-07a6df820d6f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:29.590961 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:29.591631 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:29.591966 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:29.592242 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:29.594634 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:29.717772 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:29.719391 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:29.732605 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1449/2869] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:47:33.285463 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc772e-2d3b-4ba9-ad6f-1b5251b6b055 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:33.285910 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc772e-2d3b-4ba9-ad6f-1b5251b6b055 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:33.286004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc772e-2d3b-4ba9-ad6f-1b5251b6b055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:33.336718 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8dcc772e-2d3b-4ba9-ad6f-1b5251b6b055 None None] MFA Rules not processed for user `eb0927ae32a441c7acffa98f1901a800`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:33.338007 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8dcc772e-2d3b-4ba9-ad6f-1b5251b6b055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:33.481470 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1421/2870] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:47:33.497215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:33.499070 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:33.561817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:33.616228 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc772e-2d3b-4ba9-ad6f-1b5251b6b055 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:33.642507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc772e-2d3b-4ba9-ad6f-1b5251b6b055 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:33.649159 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:33.650412 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:33.650987 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:33.651553 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:33.654890 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:33.714215 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee26ba-8d5e-42d6-857d-489b0a067e8f service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:33.723223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05d296-7308-488c-b293-b1fa6e5e8b0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:33.723995 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05d296-7308-488c-b293-b1fa6e5e8b0f service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:33.724269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05d296-7308-488c-b293-b1fa6e5e8b0f service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:33.724535 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05d296-7308-488c-b293-b1fa6e5e8b0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:33.846285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:33.849285 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:33.863712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05d296-7308-488c-b293-b1fa6e5e8b0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:33.865802 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05d296-7308-488c-b293-b1fa6e5e8b0f service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:33.879640 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1422/2871] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:47:33.884397 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1450/2872] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:47:35.383162 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:35.388562 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:35.419534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:35.466800 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012c5bf-4c68-4bcb-af0d-1a4f8bb3efc2 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:35.470745 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96be241231247b39e544529a4cd0b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:35.471576 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:35.471762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:35.471981 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:35.473978 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:35.511137 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:35.513180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:35.527212 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1423/2873] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:47:37.206859 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fcbdc-2562-4f7a-bfb5-7733ecf92764 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:37.208349 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fcbdc-2562-4f7a-bfb5-7733ecf92764 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:37.209144 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fcbdc-2562-4f7a-bfb5-7733ecf92764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:37.235893 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3e0fcbdc-2562-4f7a-bfb5-7733ecf92764 None None] MFA Rules not processed for user `0a7f0324eba3466aa985febfaf47b990`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:47:37.237907 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3e0fcbdc-2562-4f7a-bfb5-7733ecf92764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:37.255508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1451/2874] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 10 13:47:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:47:39.611530 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:39.613317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:39.629587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58a60a-196d-4993-be4f-702ced4b94cb service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:39.635027 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:39.635840 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:39.636184 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:39.636548 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] PATH_INFO: `/v3/users/411ff3f0306848ee91108343982806ca` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:39.638133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=411ff3f0306848ee91108343982806ca)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:39.640274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:39.692587 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411ff3f0306848ee91108343982806ca'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:39.703457 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411ff3f0306848ee91108343982806ca'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:39.704163 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411ff3f0306848ee91108343982806ca'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:39.714180 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411ff3f0306848ee91108343982806ca'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:39.727633 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411ff3f0306848ee91108343982806ca'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:39.736765 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1424/2875] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:47:39 2022] DELETE /identity/v3/users/411ff3f0306848ee91108343982806ca => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:47:39.954785 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fcbdc-2562-4f7a-bfb5-7733ecf92764 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:39.972409 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fcbdc-2562-4f7a-bfb5-7733ecf92764 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:40.061775 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:40.062437 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:40.062712 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:40.062981 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] PATH_INFO: `/v3/projects/b52f008781f64d09b706d478c8fe8922` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:40.064148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52f008781f64d09b706d478c8fe8922)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:40.066136 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:40.105487 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:40.121576 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52f008781f64d09b706d478c8fe8922', 'request_id': 'req-1e189d93-c3c1-496a-a3fd-92bf1702d778'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.128417 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52f008781f64d09b706d478c8fe8922', 'request_id': 'req-1e189d93-c3c1-496a-a3fd-92bf1702d778'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.136238 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52f008781f64d09b706d478c8fe8922', 'request_id': 'req-1e189d93-c3c1-496a-a3fd-92bf1702d778'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.154023 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] The token cache is being invalidate because project b52f008781f64d09b706d478c8fe8922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:47:40.155350 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.156382 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.163760 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:40.165369 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:40.166504 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1452/2876] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:47:39 2022] DELETE /identity/v3/projects/b52f008781f64d09b706d478c8fe8922 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:40.180160 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de7f6e-e5a6-4a89-9fb5-55be377fde1d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:40.275272 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:40.275908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:40.276171 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:40.276419 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] PATH_INFO: `/v3/users/a7cfcc971ac04275adc58ab8d447a1c0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:40.277659 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7cfcc971ac04275adc58ab8d447a1c0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:40.279618 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:40.328395 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:40.339794 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7cfcc971ac04275adc58ab8d447a1c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.348230 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7cfcc971ac04275adc58ab8d447a1c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.348896 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7cfcc971ac04275adc58ab8d447a1c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.356472 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7cfcc971ac04275adc58ab8d447a1c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.369937 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7cfcc971ac04275adc58ab8d447a1c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.383366 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1425/2877] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:47:40 2022] DELETE /identity/v3/users/a7cfcc971ac04275adc58ab8d447a1c0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:40.588699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:40.591032 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:40.613581 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e189d93-c3c1-496a-a3fd-92bf1702d778 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:40.702804 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:40.703439 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:40.703717 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:40.703991 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] PATH_INFO: `/v3/projects/42e30ac99cab4e50af392312a9333b5b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:40.705148 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e30ac99cab4e50af392312a9333b5b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:40.707131 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:40.772183 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e30ac99cab4e50af392312a9333b5b', 'request_id': 'req-bdd8a525-b534-466b-bceb-2c770965df51'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.779629 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e30ac99cab4e50af392312a9333b5b', 'request_id': 'req-bdd8a525-b534-466b-bceb-2c770965df51'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.787036 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e30ac99cab4e50af392312a9333b5b', 'request_id': 'req-bdd8a525-b534-466b-bceb-2c770965df51'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.810884 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] The token cache is being invalidate because project 42e30ac99cab4e50af392312a9333b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:47:40.811327 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.811863 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:47:40.813413 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1453/2878] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:47:40 2022] DELETE /identity/v3/projects/42e30ac99cab4e50af392312a9333b5b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:47:50.374815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:50.807446 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:47:55.827130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:47:55.828634 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:55.907526 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56a8fa-c246-40c0-b5b1-ec05282b8541 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:47:55.969771 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:47:55.970383 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:47:55.970654 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:47:55.970922 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:47:55.972656 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:47:55.985139 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:47:55.986762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:47:55.998567 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1426/2879] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:47:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:48:05.993813 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=58 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:48:11.984401 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:48:11.986072 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:48:12.073262 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8a525-b534-466b-bceb-2c770965df51 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:48:12.077856 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:48:12.078997 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:48:12.079199 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:48:12.079548 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:48:12.082000 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:48:12.101115 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:48:12.103974 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:48:12.117500 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1454/2880] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:48:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:48:14.953283 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:48:14.955072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:48:14.967131 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed66f2-ae3e-40ca-b732-66ce20af78f8 service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:48:14.970822 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:48:14.971435 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:48:14.971713 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:48:14.971975 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:48:14.973742 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:48:15.038970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:48:15.040401 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:48:15.050134 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1427/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:48:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:48:21.901864 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:48:21.903398 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:48:21.919704 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a56ed-1474-4c21-b811-f525088e1ffb service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:48:21.925532 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:48:21.926208 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:48:21.926583 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:48:21.926888 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:48:21.929200 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:48:21.946735 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:48:21.948407 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:48:21.965636 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1455/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:48:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:48:21.971316 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:48:21.972829 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:48:22.048406 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6914a9-2c3a-4d81-a9c2-1da4a904b55b service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:48:22.053867 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:48:22.054534 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:48:22.054791 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:48:22.055081 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] PATH_INFO: `/v3/users/0968c56b8be242918903711961b6d5c0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:48:22.056224 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0968c56b8be242918903711961b6d5c0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:48:22.058250 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:48:22.105589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0968c56b8be242918903711961b6d5c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.113827 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0968c56b8be242918903711961b6d5c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.114436 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0968c56b8be242918903711961b6d5c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.123444 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0968c56b8be242918903711961b6d5c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.136912 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-09eca9d3-8000-49d1-87a5-cc82e500b24e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0968c56b8be242918903711961b6d5c0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.142908 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1428/2883] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:48:21 2022] DELETE /identity/v3/users/0968c56b8be242918903711961b6d5c0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:48:22.330996 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:48:22.342742 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5d57-0792-4975-917a-6f24bd46fede service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:48:22.416882 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:48:22.417557 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:48:22.417837 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:48:22.418105 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] PATH_INFO: `/v3/projects/bf531c40d3f74fa6aaf3d26794dca09c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:48:22.419340 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf531c40d3f74fa6aaf3d26794dca09c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:48:22.421338 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:48:22.477375 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf531c40d3f74fa6aaf3d26794dca09c', 'request_id': 'req-e76ccfca-3d8e-4334-9d41-d241de33f9c3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.482926 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf531c40d3f74fa6aaf3d26794dca09c', 'request_id': 'req-e76ccfca-3d8e-4334-9d41-d241de33f9c3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.489452 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf531c40d3f74fa6aaf3d26794dca09c', 'request_id': 'req-e76ccfca-3d8e-4334-9d41-d241de33f9c3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.504111 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] The token cache is being invalidate because project bf531c40d3f74fa6aaf3d26794dca09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:48:22.504414 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.504933 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e76ccfca-3d8e-4334-9d41-d241de33f9c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:48:22.506508 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1456/2884] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:48:22 2022] DELETE /identity/v3/projects/bf531c40d3f74fa6aaf3d26794dca09c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:48:32.139504 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:48:32.500897 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:50:25.410422 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caa54c-2a69-40a7-bc4f-d34ce0c4ca81 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:50:25.411056 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caa54c-2a69-40a7-bc4f-d34ce0c4ca81 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:50:25.411348 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caa54c-2a69-40a7-bc4f-d34ce0c4ca81 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:50:25.412820 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1429/2885] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:50:25.419664 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14b72b-ff48-48fe-a311-12f69d524f86 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:50:25.420243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14b72b-ff48-48fe-a311-12f69d524f86 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:50:25.420243 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14b72b-ff48-48fe-a311-12f69d524f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:50:25.471651 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9b14b72b-ff48-48fe-a311-12f69d524f86 None None] MFA Rules not processed for user `234eed198ab84e95ae1a7f03785a1481`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:50:25.472850 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9b14b72b-ff48-48fe-a311-12f69d524f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:50:25.530295 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1457/2886] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:50:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:50:25.560264 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caa54c-2a69-40a7-bc4f-d34ce0c4ca81 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:50:25.562016 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-19caa54c-2a69-40a7-bc4f-d34ce0c4ca81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:50:25.576099 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caa54c-2a69-40a7-bc4f-d34ce0c4ca81 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:50:25.579589 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0980565-45f2-408a-ae1f-d735acdb06e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:50:25.580191 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980565-45f2-408a-ae1f-d735acdb06e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:50:25.580468 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980565-45f2-408a-ae1f-d735acdb06e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:50:25.580732 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980565-45f2-408a-ae1f-d735acdb06e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:50:25.587970 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0980565-45f2-408a-ae1f-d735acdb06e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:50:25.589458 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0980565-45f2-408a-ae1f-d735acdb06e1 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:50:25.597214 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1430/2887] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:50:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:50:25.686677 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77baa4-26f6-465a-83e0-9bcf440f403f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:50:25.686971 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77baa4-26f6-465a-83e0-9bcf440f403f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:50:25.687226 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77baa4-26f6-465a-83e0-9bcf440f403f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:50:25.688624 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1458/2888] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 10 13:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:50:25.694529 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7112-5fb2-488c-b3d9-f9cdc9ee6e21 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:50:25.694815 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7112-5fb2-488c-b3d9-f9cdc9ee6e21 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:50:25.695091 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7112-5fb2-488c-b3d9-f9cdc9ee6e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:50:25.735824 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1a4f7112-5fb2-488c-b3d9-f9cdc9ee6e21 None None] MFA Rules not processed for user `4e395e059b5d436a8678bb302bcfafb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:50:25.737038 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1a4f7112-5fb2-488c-b3d9-f9cdc9ee6e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:50:25.806493 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1431/2889] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 10 13:50:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:50:25.817061 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77baa4-26f6-465a-83e0-9bcf440f403f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:50:25.819840 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-eb77baa4-26f6-465a-83e0-9bcf440f403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:50:25.837244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77baa4-26f6-465a-83e0-9bcf440f403f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:50:25.840751 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234eed198ab84e95ae1a7f03785a1481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:50:25.841377 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:50:25.841755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:50:25.842123 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:50:25.853085 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:50:25.854560 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:50:25.865929 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1459/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:50:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:50:28.298680 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7112-5fb2-488c-b3d9-f9cdc9ee6e21 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:50:28.300133 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1a4f7112-5fb2-488c-b3d9-f9cdc9ee6e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:50:28.319512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7112-5fb2-488c-b3d9-f9cdc9ee6e21 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:50:28.335688 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:50:28.336327 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:50:28.336599 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:50:28.336859 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:50:28.338850 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:50:28.352227 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:50:28.353880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:50:28.368015 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1432/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:50:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:50:35.860928 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=39 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:50:38.361317 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=49 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:50:50.832952 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:50:50.834412 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:50:50.923539 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf1fff-91ed-469e-acba-062f8c4cad54 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:50:50.929968 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:50:50.931049 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:50:50.931561 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:50:50.932057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:50:50.934899 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:50:50.951697 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:50:50.953330 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:50:50.965038 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1460/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:50:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:51:00.960099 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=32 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:51:50.444880 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:51:50.459072 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8f435-08fc-4dc0-aac3-06ace5367338 service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:51:50.467128 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:51:50.468022 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:51:50.468558 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:51:50.469129 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:51:50.471141 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:51:50.487593 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:51:50.491507 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:51:50.505339 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1433/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:51:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:51:52.625989 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:51:52.638597 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a698f-1c65-48c5-bd62-3b6e19d6d01e service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:51:52.644656 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e395e059b5d436a8678bb302bcfafb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:51:52.645284 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:51:52.645572 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:51:52.645841 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:51:52.647553 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:51:52.660440 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:51:52.662048 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:51:52.672790 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1461/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:51:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:51:58.715762 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:51:58.717550 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:51:58.733144 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132af006-2020-4073-91f7-4eaa15a6d3c4 service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:51:58.738135 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:51:58.738737 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:51:58.739011 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:51:58.739274 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:51:58.741279 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:51:58.754486 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:51:58.756579 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:51:58.777004 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1434/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:51:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:52:02.193039 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:52:02.194806 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:52:02.283773 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56656e94-6c2c-4945-a109-d59d1b8bc4f8 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:52:02.288314 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:52:02.288943 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:52:02.289216 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:52:02.289513 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] PATH_INFO: `/v3/users/0a7f0324eba3466aa985febfaf47b990` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:52:02.291645 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a7f0324eba3466aa985febfaf47b990)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:52:02.296269 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:52:02.347755 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a7f0324eba3466aa985febfaf47b990'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.356272 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a7f0324eba3466aa985febfaf47b990'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.357024 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a7f0324eba3466aa985febfaf47b990'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.370451 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a7f0324eba3466aa985febfaf47b990'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.371125 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a7f0324eba3466aa985febfaf47b990'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.383732 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1462/2896] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:52:02 2022] DELETE /identity/v3/users/0a7f0324eba3466aa985febfaf47b990 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:52:02.563273 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:52:02.577773 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b097af-0891-47b5-a5f6-154892c47804 service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:52:02.642786 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:52:02.643415 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:52:02.643726 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:52:02.643996 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] PATH_INFO: `/v3/projects/15d732726c004517ad7e771af3f3b805` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:52:02.645109 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d732726c004517ad7e771af3f3b805)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:52:02.647041 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:52:02.694482 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d732726c004517ad7e771af3f3b805', 'request_id': 'req-90fda204-f02d-4257-9618-9380e9dbd45c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.699833 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d732726c004517ad7e771af3f3b805', 'request_id': 'req-90fda204-f02d-4257-9618-9380e9dbd45c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.706623 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d732726c004517ad7e771af3f3b805', 'request_id': 'req-90fda204-f02d-4257-9618-9380e9dbd45c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.720284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] The token cache is being invalidate because project 15d732726c004517ad7e771af3f3b805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:52:02.720614 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.721434 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.724313 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1435/2897] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:52:02 2022] DELETE /identity/v3/projects/15d732726c004517ad7e771af3f3b805 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:52:02.728567 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:52:02.730207 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:52:02.745223 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c48db-0dd9-4900-bcac-3cc61e01fc78 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:52:02.810900 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:52:02.811134 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:52:02.811817 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:52:02.812057 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] PATH_INFO: `/v3/users/eb0927ae32a441c7acffa98f1901a800` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:52:02.814129 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0927ae32a441c7acffa98f1901a800)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:52:02.815244 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:52:02.821891 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:52:02.870317 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0927ae32a441c7acffa98f1901a800'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.879065 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0927ae32a441c7acffa98f1901a800'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.879777 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0927ae32a441c7acffa98f1901a800'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.895356 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0927ae32a441c7acffa98f1901a800'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.896064 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0927ae32a441c7acffa98f1901a800'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:02.908709 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1463/2898] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 10 13:52:02 2022] DELETE /identity/v3/users/eb0927ae32a441c7acffa98f1901a800 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:52:03.086865 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:52:03.088582 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:52:03.106173 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fda204-f02d-4257-9618-9380e9dbd45c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:52:03.172542 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d95623897cc4c628d91416fe290bfa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cfe98f92774b529ebade94e29001a3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:52:03.173219 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:52:03.173524 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:52:03.173790 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] PATH_INFO: `/v3/projects/8dfcadebf5bc4ef2b2469e269f5a743f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:52:03.175169 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dfcadebf5bc4ef2b2469e269f5a743f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:52:03.178101 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:52:03.187259 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:52:03.233284 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dfcadebf5bc4ef2b2469e269f5a743f', 'request_id': 'req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:03.238855 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dfcadebf5bc4ef2b2469e269f5a743f', 'request_id': 'req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:03.245512 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dfcadebf5bc4ef2b2469e269f5a743f', 'request_id': 'req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:03.259130 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] The token cache is being invalidate because project 8dfcadebf5bc4ef2b2469e269f5a743f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:52:03.259477 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:03.260121 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-bc1ce8ee-435d-4b56-bea5-c1fa9ff86929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:52:03.261724 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1436/2899] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 10 13:52:03 2022] DELETE /identity/v3/projects/8dfcadebf5bc4ef2b2469e269f5a743f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:52:12.905181 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:52:13.255990 dv-cinder-852009-3-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:55:57.798209 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:55:57.799719 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:55:57.951593 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782752ff-d7ed-4b10-ae73-e27b99b02a89 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:55:58.015578 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf0871-f815-4172-b5fc-4e439f6cbe3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a31a4ad1f947308df7e4de047e85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5c8005917984c0d8397df48b0822d92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:55:58.016202 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf0871-f815-4172-b5fc-4e439f6cbe3e service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:55:58.016468 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf0871-f815-4172-b5fc-4e439f6cbe3e service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:55:58.016734 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf0871-f815-4172-b5fc-4e439f6cbe3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:55:58.019189 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dbcf0871-f815-4172-b5fc-4e439f6cbe3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:55:58.031486 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf0871-f815-4172-b5fc-4e439f6cbe3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:55:58.033078 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf0871-f815-4172-b5fc-4e439f6cbe3e service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:55:58.043705 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1464/2900] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 10 13:55:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:56:08.038699 dv-cinder-852009-3-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}